When running the TensorFlow's object-detection model inference with my own dataset, I got the following error (with Nvidia GTX 970, CUDA 8, TensorFlow 1.2.1 through pip, and Ubuntu 16.04):
2017-08-15 21:18:06.254989: E tensorflow/stream_executor/cuda/cuda_dnn.cc:359] could not create cudnn handle: CUDNN_STATUS_INTERNAL_ERROR
2017-08-15 21:18:06.255027: E tensorflow/stream_executor/cuda/cuda_dnn.cc:326] could not destroy cudnn handle: CUDNN_STATUS_BAD_PARAM
2017-08-15 21:18:06.255036: F tensorflow/core/kernels/conv_ops.cc:671] Check failed: stream->parent()->GetConvolveAlgorithms(&algorithms)
Solution from
https://github.com/tensorflow/tensorflow/issues/6698, by enabling the GPU's allow_growth flag, solved my problem:
config = tf.ConfigProto()
config.gpu_options.allow_growth = True
sess = tf.Session(config=config)
That's all :)
OMG MATE THANK YOU SO MUCH, I HATE A 970 TOO, I THINK IT'S A GENERAL PROBLEM !!!!!!!!!!!!!! THANK YOU VERY MUCH !!!!!!!!!!!!!
ReplyDeleteGREETINGS from Portugal!!!!
This solution works ! Thanks,
ReplyDeletethanks
ReplyDelete