Apache Spark is incredibly powerful, but anyone who has worked with it long enough knows the feeling:
Why is this job suddenly slower today?
Why are executors running out of memory?
Why is one stage taking 90% of the runtime?
What exactly is Spark doing behind the scenes?
After a successful first edition in 2025, CERN PGDay returns in 2026 as a regular gathering for PostgreSQL users and enthusiasts in Suisse Romande (western Switzerland). Co-organized by CERN and SwissPUG, the event offers a chance to connect, share ideas, and exchange experiences in the vibrant Geneva region — home to many international organizations across the public, private, and scientific sectors.
TL;DR: Apache Spark 4 lets you build first-class data sources in pure Python. If your reader yields Arrow RecordBatch objects, Spark ingests them with reduced Python↔JVM serialization overhead. I used this to ship a ROOT data format reader for PySpark.
I recently chased an Oracle
CERN PGDay 2025 builds on the experience of past PostgreSQL events at CERN and a newly established collaboration with SwissPUG. It will create an opportunity for PostgreSQL users and enthusiasts to meet in the French-speaking part of Switzerland in order to network and exchange their experiences.
Apache Spark is renowned for its speed and efficiency in handling large-scale data processing. However, optimizing Spark to achieve maximum performance requires a precise understanding of its inner workings. This blog post will guide you through establishing a Spark Performance Lab with essential tools and techniques aimed at enhancing Spark performance through detailed metrics analysis.
In the ever-evolving landscape of big data, Apache Spark and Apache Parquet continue to introduce game-changing features.
TL;DR Explore a step-by-step example of troubleshooting Apache Spark job performance using flame graph visualization and profiling. Discover the seamless integration of Grafana Pyroscope with Spark for streamlined data collection and visualization.
Dive into a comprehensive load-testing exploration using Apache Spark with CPU-intensive workloads.
This blog post is about building a getting-started example for semantic search using vector databases and large language models (LLMs), an example of retrieval augmented generation (RAG) architecture. You can find the accompanying notebook at this link. See also the SWAN gallery.
Are you looking at some resources to get you up to speed with popular Deep Learning and Data processing frameworks? This blog entry provides a curated collection of notebooks that will help you kickstart your journey.
You can find the notebooks at this link. See also the SWAN gallery.
This document describes some basic CPU load testing
We are in a golden age for distributed data processing, with an abundance of tools and solutions emerging from industry and open source. High Energy Physics (HEP) experiments at the LHC stand to profit from all this progress, as they are data-intensive operations with several hundreds of Petabytes of data to collect and process.
While working on a data set it is important that it stays easily and quickly accessible. Hibernate second-level caching with Coherence offers applications a resource optimized solution that keeps frequently used data in memory, by distributing it among different application instances, or sharing it with one or more dedicated cache machines. This article describes the knowledge that we gained through using the Oracle Coherence Community Edition for Hibernate second-level caching and gives a general overview of how this product can be used with Java applications running on Kubernetes.
Author: Viktor Kozlovszky
In today’s post I will describe the process of integrating OIDC implicit flow with ORDS running on Tomcat against Keycloak service. May sound complicated, but we’ll break it down into individual components so we know what we’re talking about.
TLDR; Apache Spark 3.0 comes with many improvements, including new features for memory monitoring.
The views expressed in this blog are those of the authors and cannot be regarded as representing CERN’s official position.
CERN update, Quantum Diaries, Careers at CERN
Christian Antognini, Karl Arao, Martin Bach, Mark Bobak, Wolfgang Breitling, Doug Burns, Kevin Closson, Cloudera blog, Wim Coekaerts, Bertrand Drouvot, Enkitec blog, Pete Finnigan, Richard Foote, Randolf Geist, Marco Gralike, Brendan Gregg, Kyle Hailey, Tim Hall, Uwe Hesse, Frits Hoogland, Hortonworks blog, Integrity Oracle Security, Tom Kyte, Adam Leventhal, Jonathan Lewis, Cary Millsap, James Morle, Karen Morton, Arup Nanda, Mogens Nørgaard, Oracle The Data Warehouse insider, Oracle Enterprise Manager, Oracle Linux blog, Oracle Multitenant, Oracle Optimizer blog, Oracle R technologies, Oracle Upgrade blog, Oracle Virtualization blog, Kerry Osborne, Tanel Poder, Planet PostgreSQL, Kellyn Pot'Vin, Pythian blog, Greg Rahn, Mark Rittman, Riyaj Shamsudeen, Chen Shapira, Carlos Sierra, Szymon Skorupinski