We'll give an overview of how deep-learning in healthcare can be utilized beyond medical imaging, if applied to clinical decision support and medical asset management. Deep learning is capable of addressing many, if not all, main challenges for care givers: information overflow, work overload, impacted accuracy due to data constrains, optimism bias, and optimal utilization of medical equipment. This needs to involve multiple data sources, and deals with data harmonization, semantic interoperability, and different health data types. Deep learning in healthcare has three main aspects: medical imaging, multi-data (structured, unstructured, streaming, etc.) based decision support, and asset utilization data.