Profiler Test Specifications
Author: Emanuel Hucka
Version: 1.0.0
Last update: 13.9.2005
Introduction:
This document maps test specifications of NetBeans profiler. The profiler is very complex
product with many features depending on many variables like platform, jdk version, type of profiled application etc..
Therefore the profiler test specification is composed from parts describing different
problems/approaches in work with the profiler.
Test Specifications
1. Application types
Profiler can work with different types of applications (projects).
Each of them has different method of execution and requires different test approaches.
The profiler supports:
- Applets
- J2SE Applications
- Junit tests
- Web applications
- EJB applications
To test this type of issues it is enough to start profiling of an application,
get any results and close the application.
Steps:
- Start profiling - j2se, web and ejb application simply set
as main project and invoke Profile|Profile Main Project,
for applets and junit tests - open an applet or junit test in editor
and invoke Profile|Profile Other|Profile File
- Set any type of profiling
- Wait for profilier to get started and profiling results are available -
Live Results button in Profiler control panel is enabled
- You can take a snapshot if it is possible
- Finish profiling by Stop button
2. JDK Versions
Profiler uses features of JVM introduced since JDK version 1.5.0. But there is a
private build of JDK 1.4.2 which includes this features too. This jdk includes in addition
possibility of 'Dynamic attaching'. This jdk is available only by nbm from profiler web pages now.
All profiler testing should be done with jdk 1.5.0. It is enough to make simple sanity test of
private jdk with emphasis on dynamic attaching (see Attach wizard).
3. Platforms
All platforms supported by NetBeans IDE should be supported by the profiler too.
Main supported platforms:
- windows
- linux
- solaris sparc
- solaris x86
Platform specific native code, different threads schedulling politics and cpu
timing on different platforms indicate importance of platform tests.
Tested can be everything plus features specific only for a platform like CPU timer for solaris.
See
Custom Profiling configuration|CPU|Basic options|Use Thread CPU timer.
4. Profiling Types
Profiler can (in reflect of users' needs) measure different types of values.
It can:
- monitor application, i.e. measure global memory usage, garbage collections,
threads' states
- measure CPU resources values, i.e. how much time a method takes,
methods call tree
- measure exact length of a piece of code
- measure memory values, i.e. objects creations, sizes and garbage collections
Try to measure an application by each type of profiling. Change profiling settings described in
Profiler Help and check if the profiler throws an error.
5. Profiling results
Measured data of each profiling type should be verified by defined tolerancy.
There should be defined and evaluate these values:
- list of threads, long changes of threads' states (about seconds) for monitoring
- length of a method in ms and number of invocations of a method for CPU profiling
- exact length of a method or piece of code for code region profiling
- number and size of objects created and collected within profiling for memory profiling
Each measurements should be provided with testing data to easy identify results issues:
Threads monitoring should identify thread names and states persisting cca 100ms
CPU profiling should measure time spent in a method (up to 10 ms) and whole call tree of each method
Code Region should measure code region length like CPU profiling
Memory profiling should count number of objects created and collected and their size
6. Outside profiling
Most of profiled application can be started and profiled in IDE with profiler
but some of them (configured servers) have to be started alone or on some remote machine.
Hence the profiler provides ways to start profiler agent separately
and attach to it by socket. There are three types of attaching:
- local attach
- remote attach
- dynamic attach
Try to attach from different platforms, check data in Attach wizard dialog.
7. Settings and others
Testing of other features:
- calibration - remove <USER_HOME>/.nbprofiler/machinedata* and invoke
Profile|Advanced Commands|Run Calibration - select JDK
- project integration - start profiling of a new project - there should be displayed ask dialog about
profiler integration to the project - build.xml is rewrtitten and nbproject/profiler-build-impl.xml is created
- snapshots - try to create a snapshot through profiling, store it and load it by Profile|Load Snapshot
- actions - stop, re-run, modify - use each action in profiler control panel
and check if each of them behaves as is described in manual (Profile|Profiler Help -- Control Panel)
- telemetry - start a profiling and invoke Telemetry buttorn from Profiler control panel, it should show measurements of gc, memory and threads
