We'll present methods and techniques for profiling CUDA Deep Neural Network (cuDNN) applications using the TAU Performance System. Given cuDNN applications that may take hours or days to execute, it's important to evaluate their performance and the library frameworks used to develop them. Attendees will learn approaches for measuring, analyzing, and tuning the configuration and performance of DNN applictions with TAU's techniques and tools. The DNN characteristics and features that can be exposed by TAU will help to debug faulty or subpar configurations, while informing the user of possible optimization for different GPU architectures. Results will be shown of known DNN benchmarks on a variety of GPUs.