Deep learning has become a popular tool for insight on problems where deterministic models don't yet exist. Recent development of deep learning frameworks using GPUs has allowed the application of deep learning to problems where fast solutions are required. The scientific community has traditionally sought to develop deterministic models to describe physical phenomena, using highly scalable systems to simulate problems with ever increasing fidelity. While many science domains have developed robust predictive methods, there are still problems lacking models that can describe observed phenomena. In many of these cases, the problem may contain unknown variables, or be fundamentally hard to solve, where the simulation cannot fully predict observations. These areas include biological systems, chaotic systems, and medical research. There are also fields where a priori models do exist, but surveying the parameter space through simulation of large datasets would have very long time-to-solutions. These areas include instrument data analysis and materials by design. We'll explore how the scientific community is using deep learning to conduct leading-edge research outside of traditional modeling techniques. We'll also explore opportunities and obstacles to scaling deep learning workloads on high performance computing systems.