Benchmarking is the process of discovering how the best performance can be obtained against previously defined benchmarks. It is a tool used in many industries including the world of IT and databases. The benchmarks are standards or measurements that may have been made available by hardware or software vendors in their quest to sway your purchasing decisions. An organization often engages in benchmarking when introducing a new software or hardware platform into its environment. As such, it can be seen as a rarely performed yet vitally important activity that can be thrust upon an innocent DBA.
Due to the relative infrequency of database benchmarking, the average database team may lack familiarity with the process. This can be problematic, as there may be substantial financial decisions made based on the benchmarking results. As a DBA, you don’t want to be responsible for the wrong hardware or software purchasing choices due to ineffective benchmarking.
IDERA has produced a webcast presented by Oracle Ace Bert Scalzo that looks at the problems that can plague the benchmarking process and provides some advice that will increase the success of the process. Bert’s unique style and depth of knowledge make this video required viewing for database teams preparing to engage in benchmarking. This post is intended as an overview of the concepts and information contained in the webcast.
It’s Hard to Get it Right
For many reasons, it is difficult to perform effective benchmarking without extensive planning and experience. Unfortunately, experience may be in short supply due to the nature of the beast. Planning, on the other hand, is an essential component that can be adequately addressed with proper guidance. Let’s take a look at some of the major points that have been identified as the main impediments to viable and productive benchmarking.
Not having the right resources allocated to the benchmarking effort can doom it to failure before it even starts. Here are some of the ingredients that are required for the process to increase its chances of success.
Engaging the right personnel is vitally important. Using a business analyst will not produce the same level of results as will employing a skilled DBA. The perfect candidate is a DBA with a strong knowledge of the internals of the database being benchmarked as well as the operating system, networking, and storage that support it.
Knowledge regarding the benchmark in question is equally important when preparing to engage in the process. Industry-standard benchmarks available at TPC.org offer detailed information regarding the specific tests that will be run to produce the results. Without properly understanding these details it is impossible to configure your systems in a manner that will provide viable results. DBAs need to know what to measure and why it is important.
Not allocating enough time for the benchmarking process is a problem that plagues many organizations. It takes time, much like brewing your favorite craft beer. It is unreasonable to expect valid benchmarking results unless a minimum time frame of several months is provided. One of the main reasons for a failed benchmarking effort is trying to complete the procedure with insufficient time resources.
Another reason for failed benchmarking attempts is a lack of hardware and software resources to properly model the system under review. The software needs to allow the team the flexibility to quickly modify parameters and configuration settings as the benchmarking proceeds. Dramatic changes in benchmarking results can be seen when modifying parameters. This point harkens back to the importance of apportioning enough time for the process.
Focusing on the correct metrics will inform the success of the benchmarking initiative. For instance, don’t focus on metrics such as transactions per second to the exclusion of average response time. The latter is more important to your users as it directly impacts the usability of the system.
These are just a few of the points that are discussed in this extremely informative webcast. It’s highly recommended for any team preparing to engage in benchmarking no matter their level of experience in the process. It’s likely that you will recognize an issue that has escaped your attention and that will enable your efforts to be more successful.
You Need the Right Tools
Combining the right person with benchmarking, monitoring, and database administration tools are the keys to success. IDERA’s DB PowerStudio offers a tool to perform database administration, change management, and database tuning on multiple platforms. It can be an essential part of the benchmarking process in several ways.
DB PowerStudio has several components that directly address the needs of benchmarking. Tuning database parameters and tracking them efficiently is critically important as the process proceeds. Tweaking the database to uncover enhanced performance is a vital part of conducting valid benchmarking tests. A change manager that lets you quickly roll out and track these changes will save time and effort over the longterm process. With the right tools and the information obtained in the webcast, you are on your way to successful benchmarking.