Comments for Hack MySQL Blog

Comment on New algorithm for calculating 95 percentile by Daniel Nichter

Daniel Nichter — Tue, 30 Aug 2011 14:21:11 +0000

Justin, I think 21k-1-spike5-1 is off because it’s a perfect edge case. As the filename kind of suggests, the values are:

10k @ 1s
1k @ 5s
10k @ 1s

So the 5s spike values are exactly 5% of total values (1k/20k). That make the 95%/5% edge something like 1s/5s. But the 5s values aren’t all 5s, some are 2, 3, or 4. So the edge is more like 1s/2s. Then (I should have probably made this explicit in the post), 95% here is actually “95% of values less than or equal to X” whereas technically a percentile should be just “less than X”. Using <= is inherited from mk-query-digest, but perhaps I should re-examine if that's correct.

Comment on New algorithm for calculating 95 percentile by Daniel Nichter

Daniel Nichter — Tue, 30 Aug 2011 14:06:32 +0000

Thanks for taking a look Roland. :-)

Comment on New algorithm for calculating 95 percentile by Justin Rovang

Justin Rovang — Tue, 30 Aug 2011 13:46:02 +0000

Is there a reason or conjecture to why the 21k-1-spike5-1 test was so far off?

Comment on New algorithm for calculating 95 percentile by Roland Bouman

Roland Bouman — Tue, 30 Aug 2011 12:37:33 +0000

Hi Daniel!

wow – this is great, very interesting stuff :)

I’m now downloading your code to check out the algorithm – I’m really curious how it works.

Oh, to get a quick grasp of the result, I calculated the correlation between (REAL_95,OLD_95) and (REAL_95,NEW_95), and the results are 0,9998926355 and 0,9999572588 respectively – I think it’s safe to say both are accurate enough, and it seels the new one is even a bit more accurate than the old one (for this dataset).

Exciting :)

Comment on MySQL tools article by Daniel Nichter

Daniel Nichter — Wed, 10 Aug 2011 15:56:59 +0000

I’m not aware of any tools for studying MySQL source code, but there is http://forge.mysql.com/wiki/MySQL_Internals.

Comment on MySQL tools article by whitepoplar

whitepoplar — Wed, 10 Aug 2011 01:29:21 +0000

hello

would you like to tell us that is there any good tools for studying MySQL source codes.

As i know ,gdb..and vim.

Comment on MySQL tools article by Shlomi Noach

Shlomi Noach — Sun, 07 Aug 2011 05:27:35 +0000

Daniel,
Thank you! I’m not good with publicizing, but apparently you’re doing the work for me :)

Comment on MySQL tools article by Daniel Nichter

Daniel Nichter — Tue, 02 Aug 2011 16:10:35 +0000

Shlomi has a good thing going with oak-security-audit. He should publicize it more.

Comment on MySQL tools article by George

George — Tue, 02 Aug 2011 16:02:54 +0000

Thanks Daniel, openark kit security audit is a new one for me thanks for the introduction. Added it to my tools I use including my fav mysqlreport :)

Comment on MySQL tools article by Daniel Nichter

Daniel Nichter — Tue, 02 Aug 2011 14:45:05 +0000

Thanks, Daniel. MySQL Sandbox was in my original list of possibilities, but I had to narrow it down to 10 tools and I felt MySQL Sandbox was not a tool strictly speaking because a tool does some job on/to/for MySQL but MySQL Sandbox _is_ MySQL.