Hack MySQL

Archive for the ‘Xcode’ tag

Compiling Drizzle 7 on Mac OS X 10.6

with 3 comments

Drizzle 7 GA has been released, so I wanted to compile and test it on my Mac running OS X 10.6.7.  Since Drizzle 7 is new, Mac binaries are not available yet.  I’ve compiled MySQL from source more times than I can remember, and Drizzle was forked from MySQL, so I expected the build process to be similar and pain-free, and for the most part it was.  I did not use MacPorts or Homebrew for various reasons, mainly because I know that I will compile, tweak and recompile Drizzle often while hacking on it.  Also, the blog post  Drizzle in the Snow is about building Drizzle on Mac OS X, but it’s out of date (published September 1, 2009).  Thus the need for this blog post.

After describing how I compiled Drizzle 7 on my Mac, I list several suggestions for the Drizzle developers/maintainers based on my experience.

I began, of course, by downloading and extracting the tarball from drizzle.org.  Then I proceeded to read the fine manual, even though the build was surely the standard trio: configure, make, make install.  There are three docs about compiling Drizzle: the README in the tarball, the online Drizzle documentation, and the DrizzleWiki.  The README lists that standard trio and refers to the online docu.  The online docu also lists the standard trio but, of primary importance when compiling from source, it also lists dependencies.  The wiki page on compiling also refers to the online docu, and if you Google for “compile drizzle mac” you’ll find other wiki pages but they are all out of date (e.g. one still says you need to compile libdrizzle first; this is no longer true: libdrizzle comes with Drizzle).  So, the online docu is the best source because we need to know Drizzle’s dependencies.

The online docu does not have specific instructions for compiling Drizzle from source on Mac OS X.  On this platform, the number one pre-requisite is Xcode: Apple’s development package which installs all the basic programs for compiling code (e.g. g++, make, etc.)  I’m using Xcode 3.2 because it’s free (whereas Xcode 4 cost a few dollars).  So, Xcode must be installed first before any other packages can be compiled (but if you already develop on a Mac, you probably already knew this).

From the list of dependencies, I had to download and install boost, gettext, intltool, and protobuf.  These last three are really simple: configure, make, make check, make install.  I did make check because although Mac OS X is a Unix platform, it has its quirks. Those three packages compiled, tested and installed effortlessly, all using the prefix /usr/local/. boost, however, required more attention.

To start, I just followed boost’s docu page Getting Started on Unix Variants, but it didn’t work as easily as the other packages. First, since Drizzle lists only a few boost libs as dependencies, I only compiled those:

./bootstrap.sh \
  --with-libraries=date_time,filesystem,iostreams,program_options,regex,test,thread

When that finishes, it tells me:

Bootstrapping is done. To build, run:

    ./bjam

That is different from the docu which says to run bjam install. Running just bjam does compile the libs, but it doesn’t install them. Running with the install argument makes bjam hang. So I tried bjam --install which compiles the libs but still doesn’t install the libs. Furthermore, the compiled libs did not have -mt suffixes to indicate that they where multi-threading. I noticed that Drizzle configure checks first for libs with the -mt suffix, so I really wanted to make my boost libs multi-threading. After asking around then filing a boost bug about this, I was informed that the docu was out of date and that I needed to use --layout=tagged. So, finally, to compile and manually install the boost libs for Drizzle I did (all commands):

bootstrap.sh \
  --with-libraries=date_time,filesystem,iostreams,program_options,regex,test,thread
bjam --layout=tagged
sudo cp stage/lib/* /usr/local/lib/
sudo mv boost /usr/local/include/

After that, /usr/local/include/boost/ has the boost header files and /usr/local/lib/ has boost libs like:

/usr/local/lib$ ls libboost*
libboost_date_time-mt.a                libboost_regex-mt.a
libboost_date_time-mt.dylib*           libboost_regex-mt.dylib*
libboost_filesystem-mt.a               libboost_system-mt.a
libboost_filesystem-mt.dylib*          libboost_system-mt.dylib*
libboost_iostreams-mt.a                libboost_test_exec_monitor-mt.a
libboost_iostreams-mt.dylib*           libboost_thread-mt.a
libboost_prg_exec_monitor-mt.a         libboost_thread-mt.dylib*
libboost_prg_exec_monitor-mt.dylib*    libboost_unit_test_framework-mt.a
libboost_program_options-mt.a          libboost_unit_test_framework-mt.dylib*
libboost_program_options-mt.dylib*

Now Drizzle configure finishes, it doesn’t die saying it can’t find a library it needs. Furthermore, I see that it finds the -mt boost libs, so I hope that makes it happy (because, after all, one selling point for Drizzle is massive concurrency, so I don’t want to build it without being sure it’s using multi-threading libs). But there’s a problem: configure finishes by saying:

Configuration summary for drizzle7 version 7 drizzle

   * Installation prefix:       /usr/local
   * System type:               apple-darwin10.7.0
   * pandora-build version:     0.175
   * Host CPU:                  i386
   * C Compiler:                i686-apple-darwin10-gcc-4.2.1 (GCC) 4.2.1 (Apple Inc. build 5664)
   * C++ Compiler:              i686-apple-darwin10-g++-4.2.1 (GCC) 4.2.1 (Apple Inc. build 5664)
   * Assertions enabled:        yes
   * Debug enabled:             no
   * Profiling enabled:         no
   * Coverage enabled:          no
   * Warnings as failure:       no

My MacBook Pro has an Intel Core 2 Duo, so the CPU is definitely not i386; it should be x86_64. I’m not a build expert, but I think this information is detected by config.guess by looking at uname values, which on my system returns:

$ uname -a
Darwin MacBook-Pro.local 10.7.0 Darwin Kernel Version 10.7.0:
Sat Jan 29 15:17:16 PST 2011; root:xnu-1504.9.37~1/RELEASE_I386 i386

That probably explains why configure detects the CPU has i386 (and the OS version as 10.7). So to correct this, I ran: configure --build=x86_64-apple-darwin10.6 --prefix=/opt/drizzle7. (I added --prefix because I want to install it elsewhere.) That gives me the results I expect:

Configuration summary for drizzle7 version 7 drizzle

   * Installation prefix:       /opt/drizzle7
   * System type:               apple-darwin10.6
   * pandora-build version:     0.175
   * Host CPU:                  x86_64
   * C Compiler:                i686-apple-darwin10-gcc-4.2.1 (GCC) 4.2.1 (Apple Inc. build 5664)
   * C++ Compiler:              i686-apple-darwin10-g++-4.2.1 (GCC) 4.2.1 (Apple Inc. build 5664)
   * Assertions enabled:        yes
   * Debug enabled:             no
   * Profiling enabled:         no
   * Coverage enabled:          no
   * Warnings as failure:       no

Again, I’m no build expert, but I hope this has an affect (i.e. the code is compiled with modern optimizations). If nothing else, it gives me piece of mind. So I run make and after some time Drizzle compiles without any problems. More importantly, I run make test and the result:

All 557 tests were successful.
The servers were restarted 54 times
Spent 242.179 of 386 seconds executing testcases

Congratulations to the Drizzle developers! All tests passing straight out of the box on a completely foreign system (i.e. mine) is an accomplishment.

Finally, make install puts everything in the prefix I chose (/opt/drizzle7):

/opt/drizzle7$ ls *
bin:
drizzle*       drizzleadmin*  drizzledump*   drizzleimport* drizzleslap*

include:
drizzle7/       libdrizzle-1.0/

lib:
drizzle7/                       libdrizzledmessage.0.dylib*
libdrizzle.1.1.0.dylib@         libdrizzledmessage.dylib@
libdrizzle.1.dylib*             libdrizzledmessage.la*
libdrizzle.dylib@               locale/
libdrizzle.la*                  pkgconfig/
libdrizzledmessage.0.0.0.dylib@

sbin:
drizzled@  drizzled7*

share:
man/

Now to run Drizzle for the “first” time (actually I compiled and ran Drizzle last year on my PC/Ubuntu machine, so this is only my first time on my Mac). I use the command line specified in the README:

/opt/drizzle7$ ./sbin/drizzled \
   --no-defaults \
   --port=3306 \
   --basedir=/opt/drizzle7 \
   --datadir=/tmp/drizzle7/data >> /tmp/drizzle7/drizzled.err 2>&1 &
[1] 72197

/opt/drizzle7$
[1]+  Exit 1  ./sbin/drizzled [snip] >> /tmp/drizzle7/drizzle.err 2>&1

/opt/drizzle7$ cat /tmp/drizzle7/drizzled.err
unknown option port
Use --help to get a list of available options

Aborting

Oh no! drizzled failed to start because the --port option doesn’t actually exist. So the README is a little out of day. No problem: just re-run it without --port and it starts without problems:

$ mysql -h 127.1
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 1
Server version: 2011.03.13 Source distribution (drizzle)

All in all, the build process was painless. (boost gave me more problems than Drizzle.) Now I can start hacking on Drizzle and comparing it to MySQL (whether or not that’s a fair comparison, people are going to do it often). After a few minutes of looking around Drizzle, I can already see that its “out of the box experience” is quite different from MySQL’s. For example, because I didn’t build Drizzle with extra plugins, the default authentication plugin is auth_all, i.e. no authentication, and the logging_query plugin isn’t available so no slow query logging, either. These are topics and considerations for other blog posts.

Based on my experience compiling Drizzle 7 GA from source on Mac OS X 10.6, here are my recommendations to the Drizzle developers/maintainers:

  1. Remove DrizzleWiki and centralize all documentation at docs.drizzle.org.
  2. Update the tarball README and/or add an INSTALL.
  3. Update docs.drizzle.org (i.e. make everything refer only to Drizzle 7).
  4. Consolidate information on docs.drizzle.org (e.g. “Software Requirements” lists some required libs, then “Minimal Requirements” lists more).
  5. When compilation instructions specific to Mac OS X are written, mention the need for Xcode.
  6. Harass the boost developers to update their documentation.
  7. Provide Mac OS X binaries.

As a developer of a much smaller project, I know this is all easier said than done.  Overall though, good job on Drizzle 7.

Written by Daniel Nichter

April 10th, 2011 at 12:37 pm

Posted in Drizzle

Tagged with , , , ,