Tag: Tools
-
Machine Learning Pipelines for High Energy Physics Using Apache Spark with BigDL and Analytics Zoo
Machine Learning Pipelines for High Energy Physics Using Apache Spark with BigDL and Analytics Zoo Topic: This post describes a data pipeline for a machine learning task of interest in high energy physics: building a particle classifier to improve event selection at the particle detectors. The pipeline is built using tools from the “Big Data
-
SparkMeasure, a tool for performance troubleshooting of Apache Spark workloads
SparkMeasure, a tool for performance troubleshooting of Apache Spark workloads SparkMeasure SparkMeasure simplifies the collection and analysis of Apache Spark task metrics data. It is also intended as a working example of how to use Spark listeners for collecting and processing Spark performance metrics. The work on sparkMeasure has been previously presented in this blog
-
Performance Analysis of a CPU-Intensive Workload in Apache Spark
Performance Analysis of a CPU-Intensive Workload in Apache Spark Topic: This post is about techniques and tools for measuring and understanding CPU-bound and memory-bound workloads in Apache Spark. You will find examples applied to studying a simple workload consisting of reading Apache Parquet files into a Spark DataFrame. Why are the topics discussed here relevant
-
IPython/Jupyter Notebooks for Oracle
IPython/Jupyter Notebooks for Oracle Topic: In this short post you can find examples of how to use IPython/Jupyter notebooks for running SQL on Oracle. IPython/Jupyter notebooks are one of the leading free platforms for data analysis, with many advantages, notably the interactive web-based interface and a large ecosystem of readily available packages for data analysis
-
Linux BPF/bcc for Oracle Tracing
Linux BPF/bcc for Oracle Tracing Topic: In this post you will find a short discussion and pointers to the code of a few sample scripts that I have written using Linux BPF/bcc and uprobes for Oracle tracing. Previous work and motivations Tools for dynamic tracing are very useful for troubleshooting and internals investigations of Oracle
-
IPython Notebooks for Querying Apache Impala
IPython Notebooks for Querying Apache Impala Topic: in this post you can find examples of how to get started with using IPython/Jupyter notebooks for querying Apache Impala. Apache Impala is an open source massively parallel processing (MPP) SQL Query Engine for Apache Hadoop. This post explores the use of IPython for querying Impala and generates
-
Extended Stack Profiling – Ideas, Tools and Comments
Extended Stack Profiling – Ideas, Tools and Comments Topic: This post provides a short summary and pointers to previous work on Extended Stack Profiling for troubleshooting and performance investigations. Understanding the workload is an important part of troubleshooting activities. We seek answers to questions like: what is the system doing, where is the time spent,
-
Oracle Wait Events Investigated With Extended Stack Profiling and Flame Graphs
Oracle Wait Events Investigated With Extended Stack Profiling and Flame Graphs Topic: this post is about investigating Oracle wait events using stack profiles and flame graphs extended with OS-process state and Oracle wait event details. Context: The case of the DB Time > CPU Time + Wait Time Oracle instrumentation provides wait event and CPU
-
Linux Kernel Stack Profiling and Flame Graphs Applied to Oracle Investigations
Linux Kernel Stack Profiling and Flame Graphs Applied to Oracle Investigations Topic: This blog post is about kernel stack profiling and visualization with flame graphs. Context: Stack profiling and flame graphs are very useful tools and techniques for troubleshooting and investigating workloads at the OS-level and understand which code path take most of the execution
-
Flame Graphs for Oracle
Flame Graphs for Oracle Topic: This post is a hands-on introduction to using on-CPU Flame Graphs for investigating Oracle workloads. This technique is about collecting and visualizing sampled stack trace data to analyze and troubleshoot Oracle processes at the OS level (in particular applied to Linux). Motivations: The techniques and tools described here can be