Tuesday, August 15, 2017

TensorFlow: could not create cudnn handle: CUDNN_STATUS_INTERNAL_ERROR

When running the TensorFlow's object-detection model inference with my own dataset, I got the following error (with Nvidia GTX 970, CUDA 8, TensorFlow 1.2.1 through pip, and Ubuntu 16.04):
2017-08-15 21:18:06.254989: E tensorflow/stream_executor/cuda/] could not create cudnn handle: CUDNN_STATUS_INTERNAL_ERROR 2017-08-15 21:18:06.255027: E tensorflow/stream_executor/cuda/] could not destroy cudnn handle: CUDNN_STATUS_BAD_PARAM 2017-08-15 21:18:06.255036: F tensorflow/core/kernels/] Check failed: stream->parent()->GetConvolveAlgorithms(&algorithms)
Solution from, by enabling the GPU's allow_growth flag, solved my problem:
config = tf.ConfigProto()
config.gpu_options.allow_growth = True
sess = tf.Session(config=config)
That's all :)

No comments:

Post a Comment