Hybrid Cloud interactive HPC Applications on Kubernetes (2020-03-19)

We just published a follow up article about our experiences at UberCloud running HPC Engineering Applications on different Clouds and on-premises using Kubernetes as middleware-stack.


Univa Grid Engine 8.4.1 Released (2016-07-19)

Univa Grid Engine 8.4.1 was released by Univa’s engineering team last week. This is the first update for the major 8.4 release which brings in main fixes as well as new capabilities.

The major feature introduced in 8.4 is native Docker support. Native Docker support of Univa Grid Engine means that Docker enabled hosts are automatically discovered (without any configuration needed by the admin) and the installed images are forwarded to the global scheduler of Univa Grid Engine. The user just needs to select a image name and then its job script or binary is executed on an selected execution host within the automatically created Docker container. Since the Docker containers are not anymore child processes of UGE and therefore not directly supervised by Univa Grid Engine (for accounting and resource limitation for example) a new process is injected in the container which supervises the job within the container as Univa Grid Engine does it for jobs outside of a container. Working directories and job spooling directories are automatically mounted inside the container.

Univa Grid Engine 8.4.1 adds now capabilities to launch containers with all the parameters you know from Docker run with a new qsub parameter called -xd (to select ports or devices for example).

Univa Grid Engine is a proven and a highly scalable (thousands of compute nodes) solution not just for traditional compute jobs, now also for Docker containers, which comes with enterprise features like accounting, reporting, access control, fair usage of compute resources through a set of policies which controls job priorities, simplification of administration and job submission through job classes, APIs, and quotas on container usage.

Update of the Univa Grid Engine Vagrant integration (2016-06-15)

I just updated the Vagrant integration for Univa Grid Engine so that it allows you to create an Univa Grid Engine 8.4 test cluster with three hosts just by calling vagrant up.

Just place the new Univa Grid Engine 8.4 Demo Packages ge-8.4.0-demo-bin-lx-amd64.tar.gz and ge-8.4.0-demo-common.tar.gz in the directory. You can download them from Univa

The referenced CentOS box was required to be updated to version 7.2, so if it is not yet cached locally by Vagrant a bigger download will follow. Also Docker is now going to be installed (and a config change needs to be done) on the hosts so that you easily can try the Docker integration by requesting images for your jobs. Also a simple script which downloads, compiles, and installs runc is in the directory.

For any kind of problems please create an issue at the github repository directly.

After the intallation process you can run your first Docker job in Univa Grid Engine:

  • vagrant up --> this might take a while
  • vagrant ssh --> you are now on the master node
  • qhost --> just showing if the cluster is up and running
  • docker pull golang:latest --> we need an image for our container
  • cd /nfs
  • warning: mounting non-fitting /etc in Docker could become dirty
  • qsub -b y -l docker,docker_images="*golang:latest*" -wd /nfs -xdv /nfs:/nfs,/etc:/etc -S /bin/sh /bin/sleep 123
  • docker ps --> it is launched locally because our image is only there / watch the created container
  • qdel --> remove the job and along with it the container
  • docker ps --> see that the container is removed
  • launch now an interactive job
  • qrsh -l docker,docker_images="*golang:latest*" -xdv /nfs:/nfs,/etc:/etc
  • you are now in the container
  • ps -ef --> see PID namespace isolation within the container

Univa Tech Days and Upcoming Univa Grid Engine Webinar (2016-06-14)

For more information about Univa Grid Engine 8.4 please also consider the upcoming webinar. Since seats are limited, please register early!

Also Univa's Tech Days are back! For more information please check out following link.

Univa Grid Engine 8.4 Release (2016-06-14)

Univa Grid Engine 8.4 is out!

You’ve probably already come across the news that my dear colleagues of the Univa Grid Engine development team released the 5th major Univa Grid Engine version in the year five after releasing the first Univa Grid Engine version 8.0.

Again, too many improvements to cover them all here in detail. If you are interested in all changes please consult the release-notes published here at univa.com.

Just a few highlights for the moment I really think they should be considered by all Grid Engine users as I think they will open many more use cases for running applications as simple as possible for the user and admins.

First of all, of course, the new native Docker support. Docker is everywhere, everyone likes to experiment with it, or use it already since a while in production. This is also true for Grid Engine users. While the previous versions of Univa Grid Engine contained some integration scripts the new 8.4 release comes with built-in out-of-the-box Docker support which couldn’t made it simpler for both, the users and the administrators.

Built-in Docker Support - How it works in a nutshell

To summarize:

  • the execution daemon automatically recognizes the docker daemon and marks the host as docker host

  • all images which are locally available are reported to the central Univa Grid Engine scheduler and treated as resources

  • when the user selects a docker image as a required resource the job is routed to an appropriate host. Then the execution daemon automatically creates an Docker container for the job and launches the given application in the container. Within the container a new component (called co-shepherd) is launched which supervises the job and adds additional flexibility to the container due to its support for prolog, epilog and other known Univa Grid Engine features within and outside of the container

That means without further configuration the users can simply run their scripts and applications by simply doing

qsub -l docker,docker_images=java:latest ./my_application.sh

Of course this can be simplified further by creating job classes for it which allows the administrator to create quotas on specific containers and configure fine grained access control by putting job classes in the queue configuration. All the flexibility you know from Univa Grid Engine jobs are applied to Docker containers at scale.

Other Features and Functionalities

There are plenty of other features and improvements. One of my favorite is certainly the new -tcon parameter for qsub. Univa Grid Engine is certainly pretty good (and scalable) with handling job arrays, i.e. jobs which are launched thousands of times but processing each time different data sets. Those job arrays can be limited with the -tc parameter in order to prevent that too many tasks of the same jobs are started in parallel. With the new -tcon parameter (task concurrency) you can now instruct the scheduler that either all or none of the tasks should be started. By using this parameter you can now co-schedule a bunch of tasks which are working together. All what needs to be done is reading out the SGE_TASK_ID environment variable in the job script and then launch the specific application. Of course they have to share all the same properties (like limits).

More functionalities were implemented around diagnosing the system, profiling and monitoring are constantly improved in each version. Meanwhile a new man page (sge_diagnostics) summarizes all the new capabilities in a single place.

For the case you are now curious about the new release and want to try it out. Please go ahead and download the new 48-core limited trial version directly from Univa.

For setting up a 3 VM test cluster you can use the Vagrant integration, but please note it is still CentOS 6.7 based - I’m going to update it soon (just change the box version in the Vagrant file). Also the VERSION string should be set to the version you have in the installation.sh script. For questions regarding that please create issues directly on github.com.

More Articles...

  1. Free Univa Webinars in April (2016-4-4)
  2. Webinar: High Performance Computing in the Cloud? (2016/02/01)
  3. Univa is Founding Member of Cloud Native Computing Foundation (2015-07-21)
  4. Univa Tech Day - Gothenburg - 17th of March (2015-03-12)
  5. Univa Tech Days 2015 (2015-02-06)
  6. Univa Grid Engine 8.2.1 Available (2014-12-15)
  7. Webinar About Using Coprocessors in Univa Grid Engine (2014-11-24)
  8. New Grid Engine Trainings (2014-07-21)
  9. Univa Interview at ISC 2014 in Leipzig (2014-07-02)
  10. Schlumberger's ECLIPSE Integrates Univa Grid Engine (2014-06-20)
  11. Webcast about Workload and Resource Management (2014-06-12)
  12. Univa Grid Engine meets Sahara Force India Formula 1 (2014-06-04)
  13. SoGE 8.1.7 released (2014-06-03)
  14. UniCloud Explained in May (2014-04-28)
  15. The cgroups Grid Engine Webinar Available for Download (2014-04-12)
  16. Free Webinar about Univa UniSight on April, 16th 2014 (2014-04-12)
  17. Univa Grid Engine Forums 2014 (2014-02-18)
  18. This Wednesday: Webinar about Upgrading to Univa Grid Engine (2014-03-03)
  19. Tomorrow: Free Webinar about cgroup integration in Univa Grid Engine (2014-02-24)
  20. Grid Engine Training Locations for 2014 (2014-02-05)
  21. Univa Grid Engine 8.1.7 Released (2014-01-15)
  22. Article about 20th Anniversary of Grid Engine (2013-11-21)
  23. Univa got Editors' Choice Award for Top 5 Vendors to Watch (2013-11-20)
  24. Visit Univa at Supercomputing 2013 in Denver (2013-11-16)
  25. Slidecast about Univa and Aquistion of Grid Engine Assets from Oracle (2013-11-06)
  26. Great Day for Grid Engine and Univa - Univa got Copyrights of Grid Engine Code from Oracle (2013-10-23)
  27. Univa at HEPiX Fall 2013 Workshop (2013-10-17)
  28. Grid Engine Forum 2013 and Grid Engine Training (2013-10-10)
  29. Univa Grid Engine 8.1.6 is out! (2013-10-9)
  30. Grid Engine Training and Forum Series Continues 2 (2013-10-03)
  31. Grid Engine Forum Series Continues (2013-09-25)
  32. Grid Engine Forum 2013 and Grid Engine Training (2013-08-01)
  33. Son of Grid Engine 8.1.4 Released (2013-09-06)
  34. License Orchestrator - Why License Management is important (2013-08-28)
  35. Univa Announces Partnership with MapR (2013-07-23)
  36. insideHPC Technical Computing Survey (2013-03-25)
  37. Grid Engine in 2013 (2013-03-20)
  38. Son of Grid Engine 8.1.3 released (2013-02-27)
  39. Free Archimedes / Univa Webinar in March about Shared Hadoop Infrastructures (2013-02-25)
  40. Interview with Univa CEO about Grid Engine's ARM, License Orchestrator, and Hadoop Support (2012-02-16)
  41. Univa Announces Grid Engine Support for ARM-Servers - Partnership with Calxeda (2013-02-14)
  42. A First Outlook on the Univa Grid Engine License Orchestrator (2012-02-13)
  43. Univa Announces Grid Engine 8.1.3 at SC 2012 (2012-11-13)
  44. Grid Engine in the News: "4 Ways to Create Business Value in a Bad Economy With Infrastructure Transformation" (2012-10-26)
  45. Grid Engine in the News: "Big data projects: Is the hardware infrastructure overlooked?" (2012-10-18)
  46. Interested in Grid Engine? Join us on 1./2. October 2012 in Regensburg (Germany)
  47. Univa is Hiring!
  48. Grid Engine in the News: "Managing MapReduce Applications in a Shared Infrastructure" (2012-09-26)
  49. Grid Engine in the News: "Grid Engine: Running on All Four Cylinders" (2012-09-25)
  50. Son of Grid Engine 8.1.2 released
  51. Univa Grid Engine 8.1 Available for Public Download (2012-08-20)
  52. Grid Engine Evolution Summit 2012 (2012-07-25)
  53. Univa Grid Engine 8.1 Enhancement (Part 7): Univa Grid Engine Job Classes (2012-06-29)
  54. Univa is filling Sun gap in EDA industry (2012-06-29)
  55. ISC '12 - Univa Booth (2012-06-18)
  56. Interview With Univa CEO (2012-06-12)
  57. Son of Grid Engine Released Version SoG 8.1.0 with Security Fix and 5 New Bug Fixes since 8.0e (2012-06-12)
  58. Univa Grid Engine 8.1.0 in the News (2012-05-02)
  59. Univa Releases Results of HPC Survey (2012-03-15)
  60. DRMAA Version 2 Final Publication (2012-01-27)
  61. Univa Grid Engine Man Pages Added (2012-01-02)
  62. New Grid Engine Survey (2011-11-04)
  63. Univa Grid Engine 8.0.1 released
  64. Univa Grid Engine 8.0.1 reached beta state
  65. Son of Grid Engine publishes binaries
  66. Univa offers free Univa Grid Engine Trial version!
  67. Univa Grid Engine On Demand RightScale Webinar
  68. Univa Grid Engine Summer Summit 2011