In this post, I will be talking about my openlab project and internship experience.
CERN Openlab Summer Project: Achieve a 0-downtime CERN Database infrastructure
My project was about evaluating, comparing and testing Oracle Ksplice with Red Hat Kpatch.
With the rate at which security vulnerabilities are discovered these days, keeping systems up to date has become an important task. At CERN there are many critical systems for which scheduling downtime is difficult. The main goal of this project was to check whether Ksplice or Kpatch can help the CERN IT DB team, in providing services with a higher level of availability without sacrificing reliability and security. As well as to consolidate the infrastructure as data services without the need for partitioning.
Live kernel patching technique is one of the available solutions. It allows urgent security fixes to be applied without reboot or restart of the system. In the market there are many products which offer this solution, few of them are Oracle Ksplice, Red Hat Kpatch and Suse kGraft.
We started with Oracle Ksplice, installed it on an Oracle Linux server. The installation can be done either manually or using the install-uptrack script. Both the ways are simple. Initially the system had Red Hat server but the transition from it to Oracle server is effortless. Details about different available Ksplice commands and installation are well documented on the official Ksplice website [1]. Oracle frequently releases patches for Ksplice and provides support for it.
Whereas the Red Hat Kpatch requires installation of dependencies and needs to built from source. The Kpatch source code is open source, this allows to build patch module. Red Hat also offers Kpatch patches on selective Red Hat Linux versions. The Kpatch documentation [2] gives detailed information on installation, other Kpatch commands and limitations.
Both of them are useful for applying critical kernel security fixes but cannot be used as an alternative to general kernel upgrade. Ksplice is production ready but Kpatch is not, it is one of the notable difference between the two. The Ksplice patches require better description of the security fixes applied.
For testing we require a wide variety of patches to be tested against the running applications. Given the criticality of the CERN services, both the products require long term evaluation.
My summer internship experience
The whole experience at CERN was amazing. There were so many activities organized by the CERN Openlab team. We got the opportunity to attend lectures by amazing physicists and computer scientists. There were guided tours organized to the famous LHC and other CERN experiments. These lectures and tours introduced us to new topics as well as gave insights on the CERN experiments and the technologies used. We even had field trips to ETH Zurich, IBM and Open Systems, where we attended some wonderful lectures and learned about their research.
There were Openlab lightning talks and IT-DB section summer student presentations organized. They provided a great platform to talk about the project and learn about other summer student projects. I worked with the Infrastructure, Management & Storage Services (IT-DB-IMS) team. The entire team is incredible and welcoming. I am grateful to the IT-DB-IMS team for their support and valuable feedback.
I also participated in the CERN webfest which is a 48 hours hackathon. My teammates and I had a lot of fun while working on the hackathon project. There were many other innovative and interesting projects built during the hackathon. We built an online platform (CERN-Connect) to organize events, powered by the geo-localization of its users, allowing them to coordinate in real-time. We got the first place for this project. More details about the project can be checked here [3].
I have been contributing to Linux Kernel which made working on this project even more fascinating. As building patches and applying them to the running kernel using live kernel patching was different and interesting to learn about. It was a wonderful learning experience. I am thankful to my supervisor Borja Aparicio Cotarelo for answering my questions patiently, guiding and helping me with the project.
Staying together with other OpenLab summer students in the same apartment at Saint-Genis was an amazing and memorable experience. We were a big diverse family together. It was fun to cook together and learn about different cultures. We visited the nearby cities almost every weekend. I made a lot of good friends during my two months stay.
For more details on this project, please check my report [4] and CERN OpenLab Summer Students lightning talks dedicated session [5].
I would like thank to CERN Openlab and IT DB team for this wonderful internship opportunity.
References
[1] https://ksplice.oracle.com/
[2] https://github.com/dynup/kpatch
[3] https://webfest.web.cern.ch/content/cern-connect
[4] https://zenodo.org/record/1967758#.XA6RFC70mCg
[5] https://indico.cern.ch/event/727275/contributions/3100515/