Tuesday, October 26, 2010

Using multiple versions of Ruby on the same host

I've recently come across a tool called RVM or Ruby Version Manager. It enables you to run different versions of Ruby on the same host.

RVM uses git so my first step was to install git with the Homebrew package manager. Homebrew is an increasingly popular alternative to MacPorts and Fink.

Note that you'll need to install Xcode first.

/usr/bin/ruby -e "$(curl -fsSL https://raw.github.com/gist/323731)"
brew install git

Then I just followed the instructions available here.

bash < <( curl http://rvm.beginrescueend.com/releases/rvm-install-head )
source ~/.rvm/scripts/rvm
rvm install jruby,1.9.2-head

Here is what the output looks like:

info: Downloading jruby-bin-1.5.1, this may take a while depending on your connection...
info: Extracting jruby-bin-1.5.1 ...
info: Building Nailgun
info: Installing JRuby to /Users/rpark/.rvm/rubies/jruby-1.5.1
info: Importing initial gems...
info: Installing rake
info: Installing Ruby from source to: /Users/rpark/.rvm/rubies/ruby-1.9.2-head
info: Downloading source from http://svn.ruby-lang.org/repos/ruby/branches/ruby_1_9_2.
info: Copying from repo to src path...
info: Running autoconf
info: Configuring ruby-1.9.2-head, this may take a while depending on your cpu(s)...
info: Compiling ruby-1.9.2-head, this may take a while depending on your cpu(s)...
info: Installing ruby-1.9.2-head
info: Installation of ruby-1.9.2-head is complete.
info: Updating rubygems for /Users/rpark/.rvm/gems/ruby-1.9.2-head@global
info: Updating rubygems for /Users/rpark/.rvm/gems/ruby-1.9.2-head
info: adjusting shebangs for ruby-1.9.2-head (gem irb erb ri rdoc testrb rake).
info: Importing initial gems...

The install is done. When you want to switch between different versions of ruby, just use the "rvm use" command:

rvm use 1.9.2-head
11:10:23[~:15]$ which ruby
/Users/rpark/.rvm/rubies/ruby-1.9.2-head/bin/ruby
11:11:06[~:16]$ which gem
/Users/rpark/.rvm/rubies/ruby-1.9.2-head/bin/gem
rvm use jruby
11:10:01[~:11]$ which ruby
/Users/rpark/.rvm/rubies/jruby-1.5.1/bin/ruby
11:10:06[~:12]$ which gem
/Users/rpark/.rvm/rubies/jruby-1.5.1/bin/gem

Then add this to the end of ~/.profile:

[[ -s "$HOME/.rvm/scripts/rvm" ]] && . "$HOME/.rvm/scripts/rvm"

RVM allows you to use different gemsets with each ruby version. One use case is working with Rails 2.3.3 and Rails 3.0. I didn't need this feature but it's nice to know it's there.

One final step is TextMate integration. This is described here.

rvm wrapper 1.9.2-head

Then run this script:

#!/usr/bin/env bash

mkdir -p /Library/Application\ Support/TextMate/

sudo chown -R $(whoami) /Library/Application\ Support/TextMate

cd /Library/Application\ Support/TextMate/

if [[ -d Bundles/.svn ]] ; then
  cd Bundles && svn up
else
  if [[ -d Bundles ]] ; then
    mv Bundles Bundles.old
  fi
  svn co http://svn.textmate.org/trunk/Bundles
fi

exit 0

Open up Shell Variables in TextMate's Preferences -> Advanced

Set TM_RUBY to /Users/rpark/.rvm/bin/textmate_ruby

Final step:

cd /Applications/TextMate.app/Contents/SharedSupport/Support/lib/ ; mv Builder.rb Builder.rb.backup

Monday, October 4, 2010

Connecting to JDBC data source from OS X perl

Here's another blog post on a similar topic. I recently had to figure it out and I documented it already, so I thought I would share it here too so I don't forget.

Here are instructions on connecting to a JDBC data source from OS X perl. I used DBD::JDBC.

The process mainly involves setting up a local Java server that provides the front end for a JDBC driver. The Java code implements the JDBC connection to the data source. The perl code talks to the Java server on localhost via DBI so it can access the database.

Here are the steps:

1. Check out any documentation about required setup for your JDBC data source. For example, you may need to install an SSL certificate from the server to your client. You may also need to ensure that your server permits database access at all from your client or IP address.

2. Download your JDBC driver and put any .jar files into a lib directory. For example, VJDBC is a JDBC driver that enables you to establish JDBC connections over Java RMI. It comes with vjdbc.jar.

3. Install the perl modules Convert::BER, DBI, and DBD::JDBC. You can either install the modules from CPAN or install ActiveState perl. If you install from CPAN, you'll need to install Xcode first so you have a C compiler. If you use ActiveState, use PPM to install the required modules. Note that you'll still need to download the DBD::JDBC tarball from CPAN and install it manually because it doesn't show up in ppm.

4. Copy dbd_jdbc.jar from the DBD::JDBC tarball into the same lib directory as the other .jar files you've collected.

5. Download the latest version of log4j and copy log4j.jar into the same lib directory as above. You should now have at least 3 jar files: dbd_jdbc, log4j-1.2.16, vjdbc.

6. Set your CLASSPATH variable to point to your jar files. If you want this variable to be always set, add it to your .profile.

7. Startup the Java server with this command:

java -Djdbc.drivers=com.sourcefire.vjdbc.VirtualDriver -Ddbd.port=9001 com.vizdom.dbd.jdbc.Server

Note that I set the port to 9001. This is completely arbitrary and can be whatever you want it to be. But whatever you set this to, make sure you indicate this port in your perl code.

You'll probably see something like this:

log4j:WARN No appenders could be found for logger (com.vizdom.dbd.jdbc.Server).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.

You're seeing this because the code is looking for a log4j config file. Copy log4j.properties to the same directory you started the Java server so this error message won't appear.

After this, your host is now accepting JDBC connections on port 9001.

8. Here's some sample code that should work:

#!/usr/bin/perl

use strict;
use DBI;

my $user = "user";
my $pass = "password";
my $host = "10.10.10.1";
my $port = 9001;
my $url = "jdbc:vjdbc:rmi://$host:2000/VJdbc,eqe";  # Get this URL from JDBC data src

my %properties = ('user' => $user, 
                  'password' => $pass, 
                  'host.name' => $host, 
                  'host.port' => $port);

my $dsn = "dbi:JDBC:hostname=localhost;port=$port;url=$url";
my $dbh = DBI->connect($dsn, undef, undef, 
          { PrintError => 0, RaiseError => 1, jdbc_properties => \%properties })
          or die "Failed to connect: ($DBI::err) $DBI::errstr\n";
my $sql = qq/select * from table/;
my $sth = $dbh->prepare($sql);
$sth->execute();
my @row;
while (@row = $sth->fetchrow_array) {
 print join(", ", @row), "\n";
}

One last point: your URL and JDBC properties hash are configured with the IP address of the actual JDBC data source. But your data source is configured to talk with localhost. So you establish a JDBC connection to the Java server on localhost on port 9001, and the Java server in turn establishes a connection to the actual data source using the URL.

Connecting to SQL Server from OS X perl

I've been spending my coding time in the offhours working on Perl instead of Ruby. My coding time in general has been very limited, which is part of the reason for the length of time between updates. :)

My latest project is to pull data out of a Microsoft SQL Server database for analysis. I'm using perl for various reasons: I need a crossplatform environment, and I need certain libraries that only work on perl. Some of the target users for my code run on Windows.

I know that Ruby runs on Windows but it's not the platform of choice for Ruby developers. The vast majority seem to develop either on OS X or Linux. So Ruby on Windows isn't at the maturity that ActiveState perl is on Windows.

In fact, I don't even run native perl anymore on my MacBook Pro. I've switched over to ActiveState perl because I don't need to compile anything every time I want to install new CPAN libraries. And because it's ActiveState, I'm that much more confident it will work on other platforms.

The bottom line is that perl makes the most sense for what I'm trying to do. I vastly prefer ruby to perl but I don't mind working in perl when I have to.

So how to connect to SQL Server from perl? My first thought was that I could use ODBC. My research quickly took me to DBD::ODBC in CPAN. After spending some time Googling for other examples and trying to get it working, I wasn't getting anywhere.

It took me some more research until I realized that DBD::ODBC is only one piece of the whole picture. I also need an ODBC driver manager and an ODBC driver.

The two main ODBC driver managers for Unix/Linux are unixODBC and iODBC. Fortunately, iODBC is already included in OS X.

For the ODBC driver itself, I wound up using FreeTDS. Because FreeTDS is only available as source code, I had to use MacPorts to download and compile the code. MacPorts installs everything in /opt/local by default.

So here was the process:

1. Install FreeTDS. If you are using ActiveState perl as I am, you must force MacPorts to build FreeTDS as 32-bit because ActiveState is 32-bit only. If you go with the default of compiling FreeTDS as 64-bit (or x86_64) then you will get this error when you call the FreeTDS library code from ActiveState perl:

[iODBC][Driver Manager]dlopen(/opt/local/lib/libtdsodbc.so, 6): no suitable image found. Did find:
/opt/local/lib/libtdsodbc.so: mach-o, but wrong architecture (SQL-00000) at test.pl line 26

So to fix this error, edit the MacPorts configuration file:

sudo vi /opt/local/etc/macports/macports.conf

Uncomment this line:

set build_arch i386

Once you build and install FreeTDS, ensure that you can use it to talk with the database. I used tsql:

TDSVER=8.0 tsql -H 10.10.10.1 -p 1433 -U 'DOMAIN\user' -P 'password'

2. After you do this, test your ODBC driver manager and driver. I used iODBC because it comes with OS X. It comes with a utility called iodbctest. I used similar parameters to tsql when testing:

iodbctest "Driver=/opt/local/lib/libtdsodbc.so;Server=10.10.10.1;Port=1433;TDS_Version=8.0;uid=DOMAIN\user;pwd=password;Database=Database"

Note a few things:
  • Other sites tell you to make entries in odbc.ini, odbcinst.ini, or freetds.conf. If you set all parameters on the command line then you don't need to tweak these other config files.
  • The Driver parameter is set to the full path for libtdsodbc.so. This is the actual ODBC driver. The name or path may differ, depending on your OS and ODBC driver software.
  • I haven't created an ODBC data source on my Windows SQL Server host so far. Some sites say you need to do this but I found it worked without this.
3. Use the perl code with DBD::ODBC to call the database. Here's some sample code. Note how I first populate the data source ($dsn) with the parameters I used with iodbctest, and then I pass this to the DBI->connect method.
#!/usr/bin/perl

use strict;
use DBI;

my $user = 'user';
my $pass = 'password';
my $driver = "/opt/local/lib/libtdsodbc.so";
my $db_server = "10.10.10.1";
my $db_name = 'Database';
my $port = 1433;
my $tds_version = "8.0";

my $dsn = join "", ("dbi:ODBC:",
   "Driver=$driver;",
   "Server=$db_server;",
   "Port=$port;",
   "UID=$user;",
   "PWD=$pass;",
   "TDS_Version=$tds_version;",
   "Database=$db_name",
);

my $db_options = {PrintError => 1, RaiseError => 1, AutoCommit => 0, };

my $dbh = DBI->connect($dsn, $db_options);
$dbh->{LongReadLen} = 32768;
my $sql = qq/select * from table/;
my $sth = $dbh->prepare($sql);
$sth->execute();
my @row;
while (@row = $sth->fetchrow_array) {
    print join(", ", @row), "\n";
}
Note on 12/29/10: I added this after the DBI->connect: $dbh->{LongReadLen} = 32768; I was getting "Data truncated" errors when accessing ntext fields. This link was a real help and helped me solve the problem.

Thoughts on the iPad

There are many, many posts on the iPad already. I was quite surprised about the intensity of the debate of its merits, or lack thereof.

I've had my iPad for about 5 months now, and I love it. I preordered it and got it on April 30. Since then it has subtly become part of my daily routine.

Here is a list of my uses for it, ordered from most frequent to less frequent:

  1. Reading books
    • iBooks
    • GoodReader
    • Bible
  2. E-mail
  3. Looking at RSS feeds
  4. Watching movies
  5. Games
I thought it would be interesting to put together a "collage" of interesting blog posts about the iPad.

Here is an analysis of the fact that the iPad is considered to be nothing new. Its critics seize this as a point of weakness but in reality it is probably a point of strength:

Instead of praising the iPad, critics express their disappointment, because they expected more. They expected a genre buster. They expected something they’d never seen before, something beyond their imagination. Something revolutionary.

They’re disappointed that the iPad is so… well… unsurprising. “Oh,” they say. “It’s a big iPhone.”

It doesn’t matter if they utter that phrase in distaste. That little sand grain of dismissal becomes the core around which will form a pearl of understanding.

Steve knows, better maybe than anyone else, that you don’t just slap a product out there and hope it will succeed. You have to prepare people for it, first.

And it’s better that people misunderstand a product, at first, than not understand it at all.

People won’t buy a product if they can’t understand it immediately. They can’t understand it immediately if their worldview doesn’t already have a readymade place for it. And their worldview won’t have a readymade place for it, if they’ve never seen anything like it before.

Steve expertly wields the powerful tool that is the feeling of recognition.

I agree with this post here that the iPad will change the model of personal computing over time.
the iPad will change the model of personal computing -- not immediately and not for everyone, but for many millions of people the PC will begin to look like a dinosaur. One of my reasons for such a bullish view is the number of skeptics coming forward to say that the iPad is not what it is cracked up to be. Skeptics have been a reliable predictor of the next big thing -- the Internet is too insecure to allow for banking and insurance. WiFi is too expensive and slow and will fizzle. Blogging was to peak out some years ago. Social networking is a fad. The iPad is just a big iPhone.
The desktop was revolutionary in its time, with its "small" form factor and personal monitor. But there is no reason why we should be stuck with this metaphor forever. Even the laptop is just a variation on the desktop theme.


This post helps explain why the iPad has been so successful lately. You don't think you want it, and then you try it at the Apple Store or a friend's house and then you realize how much you want it.

there is a certain magic to using the iPad that's nearly impossible to convey in words - you have to touch it to believe it. the iPad becomes the app you're using. That's part of the magic. The hardware is so understated - it's just a screen, really - and because you manipulate objects and interface elements so smoothly and directly on the screen, the fact that you're using an iPad falls away. You're using the app, whatever it may be, and while you're doing so, the iPad is that app.

Some people wonder, will the iPad replace my laptop? I don't think it will. The laptop will always have a place for heavy-duty content creation, coding, etc. But for everyday use and even for travel, I have seen myself use my iPad much more. I'll post in the near future about how I've started taking my iPad on business trips instead of my laptop.

I look forward to seeing what type of brand new applications the iPad will enable in the next few years.

Sure, if the interfaces of iPad apps were just scaled-up versions of iPhone apps (like what you get if you run iPhone-only apps on the iPad), the iPad would be the technological equivalent of one of those oversized novelty checks presented to lottery winners. But what the additional pixels really allow is entirely new, richer, and more complex interactions. Beyond the more sophisticated user-interface possibilities, the iPad’s large screen opens the door for new gestures that simply wouldn’t work on a pocketable device. So can the iPad truly replace a laptop? It all depends on what you use your laptop for. The iPad isn’t going to replace a MacBook Pro anytime soon. But let’s face it: there are plenty of tasks that we currently use laptops for (checking e-mail and Twitter, surfing the Web, looking up some actor on IMDB) that don’t really tap the power of a laptop. These are the tasks the iPad is perfectly suited for.

The iPad has gotten a lot of criticism for being such a controlled environment. But game consoles are similar in how tightly controlled they are. And look how they have transformed the gaming market. I'm the last person to say that PC gaming will die any time soon, but it is only a fraction of the overall gaming market.
Able to control its hardware and software, Microsoft avoided the instability created by the endless hardware/software configurations found on PCs. Not only are users spared the pain of endless crashes, but they don't have to worry about hardware requirements when purchasing games. Rather than fret about whether one has enough Video RAM or processor speed, it's literally plug and play. Of course, that kind of stability has a price. Microsoft requires Xbox developers to register themselves, and all games must be approved by the company before they can be sold to the public. Such rigidity limits the freedom of developers to write for the platform as they see fit, but it allows Microsoft to ensure that end users get the kind of enjoyable experience that keeps them buying Xbox games. Same goes for Sony and the PlayStation, and I'm sure for Nintendo and its Wii as well. In the end, those closed gaming systems have been so effective, that they effectively killed PC gaming.

For all of the iPad's advantages, however, there are definitely some challenges. I'm very glad that Apple has recently lifted restrictions on running interpreted code on iPhones and iPads. But there are definitely some more suggestions that Apple should seriously consider.
Human-computer interaction has found a sweet spot on the iPad. It’s all the power of desktop computing, plus the valuable constraints of mobile devices, minus the limitations of both. It just makes sense. Use one for a couple hours and your desktop or laptop will seem clumsy, arbitrary, and bewildering. It is, simply, how (most) computing should be. The iPad is a beautiful, important, transformative device released under a confusing regime of questionable ethics.
  1. Apple should not charge to put applications you’ve written onto your personal iPad (or iPhone, for that matter).
  2. Apple should lift restrictions on running interpreted code on its mobile devices. Let people run Basic, Python, and Ruby interpreters on iPad and iPhone
  3. Apple should remove the concept of private APIs from its developer offerings. Give developers the same tools that Apple’s own programmers get to use.

I'll end with a final quote from one of my favorite technology analysts, Tim O'Reilly. I've been following him for years and he has generally been dead-on with his insights and predictions.
But the iPad signals more than the end of the PC era. It signals that the App Store, the first real rival to the Web as today’s dominant consumer application platform, isn’t going to be limited to smartphones. It signals that App Store-based e-commerce may replace advertising as the favored model of startup entrepreneurs. It signals that cheap sensors are ushering in an era of user interface innovation. Apple’s Achilles’s heel is that it seems to have come too late to an understanding of the key drivers of lock-in in the Internet era: not hardware, not software, but massive data services that literally get better the more people use them. Media and application syncing across iPhone and iPad is poorly thought out. MobileMe, which should be Apple’s gateway drug for lock-in to Apple services, is instead sold as an add-on to a small fraction of Apple’s customer base. If Apple wants to win, they need to understand the power of network effects in Internet services. They need to sacrifice revenue for reach, taking the opportunity of their early lead to tie users ever more closely to Apple services.