Linux Kernel Stack Profiling and Flame Graphs Applied to Oracle Investigations
Topic: This blog post is about kernel stack profiling and visualization with flame graphs.
Topic: This blog post is about kernel stack profiling and visualization with flame graphs.
Recently we were refreshing our recovery system infrastructure, by moving automatic recoveries to new servers, with big bunch of disks directly connected to each of them. Everything went fine until we started to run recoveries - they were much slower than before, even though they were running on more powerful hardware. We started investigation and found some misconfigurations, but after correcting them, performance gain was still too small.
A little bit scary title, isn't it? Please keep in mind that definitely it is neither supported nor advised method to solve your problems and you should be really careful while doing it - hopefully not on production environment. But it may sometimes happen that you end up with the situation where creating your own merge patch for Oracle database could not be as crazy idea as it sounds :).
Regular readers of our blog probably already know that for most of our databases we're using two storage layers to keep our backups - NAS volumes as a primary layer and tapes as secondary one - please check "Datafile without backups - how to restore?" for more details. If you read another post "Importance of t
Oracle Managed Files (OMF) have many advantages, but the fact that such files could coexist in the same database with manually added (and named) ones, could sometimes lead to confusion. Situation is made worse by the fact, that there is no straightforward way (at least of which I'm aware of...or rather was - please check the comment of Mikhail Velikikh) to say if the file is Oracle managed or not. Oracle documentation seems to confirm this:
I've already described how important is to test your backup strategy and restore/recovery procedures, but while doing so, you could of course encounter some problems, not really related with the recoverability as such. Recently, we've got such a problem on our recovery server, at the very beginning of an automatic restore (database name masked):
Since version 11.1 of Oracle database, there is very useful command available, allowing DBAs to easily move RMAN recovery catalog schemas between databases. Its functionality is even broader, as it also allows one catalog schema to be merged into another - either the whole schema or just the metadata of chosen databases. Command I'm writing about is of course import catalog, which I had a chance to use recently, to move our recovery catalog to the new database.
Have you ever had a problem with restoring datafiles without any backups available? It's easy, of course if you have all archived logs from the time datafile was created. Please check it here: Re-Creating Data Files When Backups Are Unavailable. Moreover, RMAN is clever enough to create empty datafile automatically during restore phase and then recover it using archived logs. So far, so good, but...
Most of you for sure know, that ability to restore data in case of failure is a primary skill for each DBA. You should always be able to restore and recover data you’re responsible for. This is an axiom. To be sure, that you’re able to do it, you should test it on regular basis. There is of course possibility to use some Oracle features, like backup ... validate or restore ...
Physical standby databases seem to be ideal candidates for offloading backups from primary ones. Instead of "wasting" resources (unless you're already using Active Data Guard for example), you could avoid affecting primary performance while backing up your database, especially if your storage is under heavy load even during normal (user- or application-generated) workload. So, if you're seeking for good reasons to convince your boss/finance department/etc.
The views expressed in this blog are those of the authors and cannot be regarded as representing CERN’s official position.
CERN update, Quantum Diaries, Careers at CERN
Christian Antognini, Karl Arao, Martin Bach, Mark Bobak, Wolfgang Breitling, Doug Burns, Kevin Closson, Cloudera blog, Wim Coekaerts, Bertrand Drouvot, Enkitec blog, Pete Finnigan, Richard Foote, Randolf Geist, Marco Gralike, Brendan Gregg, Kyle Hailey, Tim Hall, Uwe Hesse, Frits Hoogland, Hortonworks blog, Integrity Oracle Security, Tom Kyte, Adam Leventhal, Jonathan Lewis, Cary Millsap, James Morle, Karen Morton, Arup Nanda, Mogens Nørgaard, Oracle The Data Warehouse insider, Oracle Enterprise Manager, Oracle Linux blog, Oracle Multitenant, Oracle Optimizer blog, Oracle R technologies, Oracle Upgrade blog, Oracle Virtualization blog, Kerry Osborne, Tanel Poder, Planet PostgreSQL, Kellyn Pot'Vin, Pythian blog, Greg Rahn, Mark Rittman, Riyaj Shamsudeen, Chen Shapira, Carlos Sierra, Szymon Skorupinski