1N/AHistory - History of the DTraceToolkit
1N/A------------------------------------------------------------------------------
1N/A20-Apr-2005 Brendan Gregg Idea
1N/A For a while I had thought that a DTrace toolkit would be a nice
1N/A idea, but on this day it became clear. I was explaining DTrace to
1N/A an SSE from Sun (Canberra, Australia), who had a need for using
1N/A DTrace but didn't have the time to sit down and write all the
1N/A tools he was after. It simply made sense to have a DTrace toolkit
1N/A that people could download or carry around a copy to use. Some
1N/A people would write DTrace tools, others would use the toolkit.
1N/A------------------------------------------------------------------------------
1N/A15-May-2005 Brendan Gregg Version 0.30
1N/A I had discussed the idea of a DTrace toolkit with the Sun PAE guys in
1N/A Adelaide, Australia. It was making more sense now. It would be much
1N/A like the SE Toolkit, not just due to the large number of sample
1N/A scripts provided, but also due to the role it would play: few people
1N/A wrote SE Toolkit programs, more people used it as a toolkit. While
1N/A we would like a majority of Solaris users to write DTrace scripts,
1N/A the reality is that many would want to use a prewritten toolkit.
1N/A Today I created the toolkit as version 0.30, with 11 main directories,
1N/A a dozen scripts, man pages and a structure for documentation.
1N/A------------------------------------------------------------------------------
1N/A16-May-2005 Brendan Gregg OneLiners
1N/A I've been using the toolkit for a day now (wow!), and have noticed
1N/A a few problems I've been fixing. One of them was the dtrace oneliners.
1N/A seperate script files, or I'm doing an ls or find. Ok, so I've now
1N/A made each one liner a seperate script. This seems at first pretty
1N/A silly since they are oneliners and shouldn't deserve an entire script
1N/A each, but I've found having them as seperate scripts makes them far
1N/A easier to find and use. The scripts and man page for each script do
1N/A point out the fact that it's a one liner.
1N/A------------------------------------------------------------------------------
1N/A17-May-2005 Brendan Gregg Version 0.33
1N/A Version 0.33 with 33 scripts. Maybe I should make the version number
1N/A equal the script count. :) I just finished dtruss, dapptrace and
1N/A------------------------------------------------------------------------------
1N/A08-Jun-2005 Brendan Gregg Name changes.
1N/A to call it Contents after all. I've also made a symlink to it called
1N/A------------------------------------------------------------------------------
1N/A08-Jun-2005 Brendan Gregg Version 0.35
1N/A Version 0.35 with 35 scripts. Also touched up procsystime and some
1N/A man pages. Added the CDDL version 1.0.
1N/A------------------------------------------------------------------------------
1N/A09-Jun-2005 Brendan Gregg Version 0.42
1N/A Added 7 more scripts.
1N/A------------------------------------------------------------------------------
1N/A14-Jun-2005 Brendan Gregg Version 0.57
1N/A Added heaps of new scripts. Now at 57 scripts.
1N/A------------------------------------------------------------------------------
1N/A17-Jun-2005 Brendan Gregg Version 0.61
1N/A Restyled many commands.
1N/A------------------------------------------------------------------------------
1N/A28-Jun-2005 Brendan Gregg Version 0.70
1N/A Added several commands including dexplorer. Developed a few useful
1N/A variants of classic scripts while writing dexplorer, and have added
1N/A them to the toolkit (I kept wanting to run them individually but
1N/A not have to run an entire dexplorer).
1N/A------------------------------------------------------------------------------
1N/A25-Jul-2005 Brendan Gregg Version 0.77
1N/A Added
tcpsnoop.d, tcpsnoop, tcptop. Because of their addition I have
1N/A track TCP in an accurate manner. However! also because they are
1N/A complex scripts, I expect they will require maintainence for newer
1N/A versions of [Open]Solaris, as various probes may change. They will
1N/A become much more stable once a network provider has been added to
1N/A DTrace (which may be some time away).
1N/A Also added iotop, and updated a bunch of scripts. A lot of work went
1N/A into this version, although the version change doesn't reflect that
1N/A (I'm still keeping the version number == to number of scripts).
1N/A Also added rwsnoop, rwtop, and more.
1N/A------------------------------------------------------------------------------
1N/A26-Jul-2005 Brendan Gregg Version 0.82
1N/A Many new scripts added, many updates. This is a major release.
1N/A------------------------------------------------------------------------------
1N/A17-Sep-2005 Brendan Gregg Version 0.83
1N/A A few scripts have been updated so that they work better.
1N/A execsnoop, iosnoop, opensnoop and rwsnoop will be more responsive
1N/A (increased switchrate).
1N/A------------------------------------------------------------------------------
1N/A22-Sep-2005 Brendan Gregg Version 0.84
1N/A Some updates, fixed some bugs (cputimes, cpudists). Added
cpuwalk.d. 1N/A------------------------------------------------------------------------------
1N/A15-Nov-2005 Brendan Gregg Sys Admin Magazine
1N/A Ryan Matteson wrote an article on the DTraceToolkit which has been
1N/A printed in Sys Admin Magazine, December 2005. It's quite good,
1N/A and made it as the feature article - which means it will be available
1N/A online for some time. Thanks Matty, and Sys Admin Magazine!
1N/A "Observing I/O Behavior with the DTraceToolkit"
1N/A------------------------------------------------------------------------------
1N/A01-Dec-2005 Brendan Gregg Version 0.88
1N/A Many scripts were updated. Added the Apps category. I had planned
1N/A to add some key scripts, but they haven't made it out of testing yet.
1N/A------------------------------------------------------------------------------
1N/A03-Dec-2005 Brendan Gregg Version 0.89
1N/A dtrace-discuss mailing list).
1N/A------------------------------------------------------------------------------
1N/A12-Jan-2006 Brendan Gregg Version 0.92
1N/A------------------------------------------------------------------------------
1N/A09-Apr-2006 Brendan Gregg Solaris Internals 2nd Edition
1N/A In the past few months I have been contributing to Solaris Internals
1N/A 2nd Edition. This book (now two volumes) is really amazing. The 2nd
1N/A volume does use the DTraceToolkit where appropriate, and covers loads
1N/A of useful topics. While writing and reviewing material for Solaris
1N/A Internals, I've had numerous new ideas for DTrace scripts. Not only
1N/A that, but a few people have managed to send me well styled, carefully
1N/A tested, well considered DTrace scripts for inclusion in the toolkit.
1N/A------------------------------------------------------------------------------
1N/A20-Apr-2006 Brendan Gregg TCP bug fixed
1N/A Stefan Parvu sent me a bug for the tcp* scripts: on build 31+ they
1N/A error'd on the symbol SS_TCP_FAST_ACCEPT. This symbol was
1N/A renamed to SS_DIRECT (I checked the code, they are used in the
1N/A same way). Ironically, when I first wrote the scripts I had hardcoded
1N/A the value 0x00200000, then rewrote it "properly" by importing
1N/A the header files and using the symbol name. Had I been lazy and left
1N/A it hardcoded, the bug would never have eventuated. Not to worry,
1N/A it has returned to being hardcoded, so that it works on all builds
1N/A (until something else changes).
1N/A------------------------------------------------------------------------------
1N/A21-Apr-2006 Brendan Gregg Restyled - again!
1N/A I've been writing the "DTraceToolkit Style Guide", to document
1N/A the style that these scripts obey. It is quite strict, and sets
1N/A the bar fairly high. I've been warned that it may cause very few
1N/A people to ever contribute scripts, which is fine. At some point
1N/A I'll carefully explain the mentality behind this, but in a nutshell:
1N/A Users on critical production servers expect the tools to be
1N/A accurate, carefully tested, and cause no undocumented harm.
1N/A------------------------------------------------------------------------------
1N/A22-Apr-2006 Brendan Gregg Docs changes
1N/A The "Contrib" file was merged into the "Who" file. In hindsight
1N/A it is better to keep this data together than to split it up.
1N/A------------------------------------------------------------------------------
1N/A24-Apr-2006 Brendan Gregg Version 0.96
1N/A The toolkit now contains 104 scripts, however I'll keep the version
1N/A number < 1.00 until the dust has settled on these new scripts.
1N/A There is some special significance with version 1.00, it would
1N/A imply that every script had been tested for some time - not that
1N/A I've just added a few.
1N/A There is a new main directory, FS for file system related scripts.
1N/A There are some interesting scripts in there, from or based on
1N/A Solaris Internals 2nd ed, vol 2.
1N/A------------------------------------------------------------------------------
1N/A30-Sep-2007 Brendan Gregg Version 0.99
1N/A It's been a year and a half since the last release, and a lot has
1N/A happneed. Firstly, the DTraceToolkit has featured in the Prentice Hall
1N/A Solaris Performance and Tools
1N/A DTrace and mdb techniques for Solaris 10 and OpenSolaris
1N/A written by Richard McDougall, Jim Mauro and myself. It is a companion
1N/A book to "Solaris Internals 2nd edition" by Richard McDougall and
1N/A Jim Mauro. If you are serious about becomming a DTrace guru,
1N/A especially on Solaris, then please study both books. (Yes, I realize
1N/A that many people are using the DTraceToolkit because they don't have
1N/A the time or don't want to become DTrace gurus; well, so long as
1N/A you are using DTrace anyway :). The performance book was a great relief
1N/A to write - since we were able to put to print much performance wisdom
1N/A and knowledge that was begging to be documented.
1N/A Then, in late 2006 I joined an advanced products engineering team
1N/A at Sun in San Francisco, a team which includes the three members of
1N/A team DTrace. It's been a great opportunity to learn from such
1N/A engineers, and to contribute more directly to DTrace. So far my work
1N/A has included writing a JavaScript provider, integrated inet_ntoa()
1N/A style functions into DTrace, and prototying DTrace IP, TCP and UDP
1N/A Working on the network providers is good news for the DTraceToolkit,
1N/A as it will indirectly help the tcp* scripts become more stable. Yes,
1N/A those scripts have broken a few more times during the last 18 months,
1N/A sorry about that, and it will keep happening until we have stable
1N/A network providers. This is why I only ever wrote three tcp* scripts,
1N/A and not at least a dozen, which I'd really like to do.
1N/A I did leave my pile of old SPARC and x86 development servers behind
1N/A in Australia, and brought over a couple of laptops. That has made me
1N/A more dependant on Stefan for testing the toolkit - especially on SPARC.
1N/A So, it's been about 18 months since the last release, which is
1N/A mostly due to having less spare time due to moving countries and
1N/A Michelle from Sun docs has been asking for a newer version of the
1N/A DTraceToolkit for the OpenSolaris starter kit, which is why I'm
1N/A releasing this version now and not waiting a few more weeks as
1N/A I complete bug fixes.
1N/A So the good and the bad news for this release, starting with the bad,
1N/A I've added versions that should work on Solaris Nevada and OpenSolaris
1N/A for releases from around late 2007. They are likely to break again.
1N/A The real answer, as always, is for stable nework providers to be
1N/A integrated into Solaris.
1N/A Many of the exciting new language provider scripts in this release
1N/A currently require downloading, patching and compling of the language
1N/A interpreter to get working. See the Readme file in each directory
1N/A Good: many more scripts to cover the new DTrace language providers
1N/A that are available (the DTraceToolkit is now 227 scripts). Many
1N/A updates to the Notes directory. Bug fixes. Some new categories
1N/A other than for scripts: Code - for simple programs to DTrace (and
1N/A for the example files), and Snippits - for useful lumps of DTrace
1N/A code to copy-n-paste from. The man pages are also making room
1N/A for documenting both stability and supported operating systems for
1N/A each script - now that DTrace exists for MacOS X Leopard, the
1N/A DTraceToolkit will begin supporting multiple operating systems.
1N/A This can be thought of as a developer's release of the DTraceToolkit -
1N/A to help people start using DTrace with Perl, Python, Ruby, Php, Java,
1N/A JavaScript, Shell and Tcl. I've written about 15 scripts for each
1N/A language, to cover the basics and to show the way for deeper analysis.
1N/A The scripts are also similar from one language to another, having
1N/A devoleped a tried-and-tested group of scripts for analyzing real world
1N/A issues - it made sense to repeat these scripts for every language
1N/A possible. To see what I mean, try reading,
1N/A You might notice that the example files are more clearly and carefully
1N/A explained. Claire (my wife), wrote close to one hundred of them for
1N/A this release while I focused on writing and testing the scripts.
1N/A Claire has worked as a SysAdmin and as an IT instructor, and is well
1N/A skilled at explaining relavent technical details. And she can spell
1N/A much better than I can. :)
1N/A The future: I still have many new scripts and some bug fixes in mind,
1N/A as well as generally improving the Notes and Examples provided.
1N/A Hopefully it won't be too many months before you see another
1N/A release. Check here for the lastest installment,
1N/A------------------------------------------------------------------------------