Time witnesses the revolution of infrastructure changing for on-cloud services. Micro-service with container (Docker) infrastructure is widely used in production deployment. Open-source container cloud management frameworks, notably, Mesos and Kubernetes, are widely used. We'll discuss our efforts and progress to enable GPU on container clouds with Mesos and Kubernetes. For Mesos, we bring GPU-enabled Mesos/Marathon infrastructure to Bluemix China Public as commercial PaaS for third-party's cognitive services deployment. In community, we contributed our work back and completed the preliminary GPU support on Mesos 1.0 and Marathon 1.3 with both Mesos container and Docker container, in collaboration with Mesosphere and NVIDIA teams. For Kubernetes, we completed several major GPU features extension to support IBM's internal cognitive infrastructure, and prepare to contribute back to community soon.