Jan/105
Robust DBI Transaction and Connection Handling
Edit: It turns out there is already a module on CPAN that does exactly what I talked about here. Its called DBIx::Connector. I haven’t tried it yet, but it looks like the guy that wrote it designed it with input from the DBIC guys.
I’m a big fan of DBIx::Class. Among DBIC’s many great features is its superb transaction and connection handling via DBIx::Class::Storage::DBI. When I’m using raw DBI I feel like I’m missing these core components. I should always be able to expect robust transaction and connection handling.
I started playing around with creating a new CPAN module (something like DBIx::Robust, DBIx::Transaction, etc…) but each time I dove in to it I kept coming to the conclusion that DBIC’s DBI storage drivers provide everything I need and I would just be shooting myself in the foot by either porting DBIx::Class::Storage::DBI to a DBIC independent API or by writing it from scratch.
Before I go any further, let me show you how attractive and awesome DBIC’s storage layer is:
- Transactions may be nested, even without savepoints.
- Transaction savepoints can automatically be tracked allowing for reliable and incremental rollbacks.
- The appropriate DateTime::Format class is automatically determined based on the type of database.
- All database calls efficiently pass through a layer that detects stale connections that will attempt to re-connect to a database.
“There is no charge for awesomeness, or attractiveness.”
The DBIx::Class::Storage::DBI pod illustrates all this in a round-about way: “If you set AutoCommit => 0 in your connect info, then you are always in an assumed transaction between commits, and you’re telling us you’d like to manage that manually. A lot of the magic protections offered by this module will go away. We can’t protect you from exceptions due to database disconnects because we don’t know anything about how to restart your transactions. You’re on your own for handling all sorts of exceptional cases if you choose the AutoCommit => 0 path, just as you would be with raw DBI.”
This stuff is really powerful, and we should never have to work with databases in Perl without it. But, currently, it is only meant to work under DBIC, which is a shame. I’m betting that DBIx::Class::Storage::DBI could be used directly, bypassing DBIC completely. There is some DBIC stuff layered in there, but that should be ignorable.
What do the rest of you think? Has anyone considered moving this logic out in to a generic distribution that isn’t DBIC specific? Is there merit in what I’m talking about? Does anyone use DBIx::Class::Storage::DBI directly, bypassing DBIC?
Oct/090
use HTML::FormHandler;
I’ve been meaning to give HTML::FormHandler a try for quite some time now. Over the weekend I converted a form from some home-grown form validation and html generation to use HTML::FormHandler instead. HTML::FormHandler was inspired by, and quasi-forked from, Form::Processor. There are many options for processing forms, such as FormValidator::Simple (Catalyst’s default) and Data::FormValidator, to name a few.
Up until now I’ve always thought that the existing form handling/validating modules on CPAN were incredibly lacking. They all have these crazy and inconsistent APIs that in the end limit what you are able to do. I don’t expect a form validation API to handle every possible situation, but I do expect it to provide a consistent API that is well documented and allows me to easily extend if I need additional functionality. Plus none of them use Moose (AFAIK), which isn’t just me being a Moose evangelist, its also practical because Moose provides mechanisms to extend functionality via roles and to provide reusable validation with MooseX::Types.
HTML::FormHandler solves these issues. HTML::FormHandler can be a complete end-to-end solution for declaring form fields and types, applying server-side validation, generating the HTML for a form, and applying form submits to the database. Almost everything and the kitchen sink is there, except client-side form validation, which can be added separately of course. Normally a “do everything” package scares me, especially for something like form validation. But, Gerda Shank (the author of HTML::FormHandler) did a great job splitting out the various pieces of HTML::FormHandler so that any piece of the form handling process can be subverted to your own needs, and there is copious amounts of documentation explaining how to do so.
The HTML::FormHandler::Manual::Intro does a great job introducing HTML::FormHandler.
There are a few downsides to this module’s current state. It is relatively new and is still stabilizing. For example, just recently the results of form validation were separated from the form object itself. This is a huge fundamental change, but a necessary one. Also, the concept of widgets was recently introduced. Widgets are a great way to extend and customize the display of form fields, and the form as a whole. But, this was just introduced and is still experimental, and has some rough edges. Despite this, the widget design is very promising and I ended up using it myself (thanks to HTML::FormHandler::Manual::Rendering) as my preferred rendering method.
Enjoy.
Sep/093
Top 10 Reasons to use Perl
You should use Perl because…
10. Real Perl coders don’t have fohawks.
9. Perl has nothing to do with Ryan Seacrest.
8. Beer improves Perl.
7. It comes with a 20-sided die for all that TIMTOWTDI.
6. It has nothing to do with PHP.
5. Captain Picard writes perl.
4. There is no better place to get Moo seX.
3. It grows on you.
2. IE8 was not written in Perl.
… drum roll …
1. You can write this.
Sep/092
IE8, another screwed up version.
I was really hoping that Internet Explorer version 8 (IE8) would finally be the version of IE where Microsoft stepped up to the plate and produced a fast browser, with few bugs, less Microsoft-only features, and a good and modern UI.
I’m the kind of guy that holds judgment for a very long time, sometimes too long. So it comes with great sadness that I announce that IE8 is not the browser I was hoping for. It is riddled with bugs, the rendering engine crashes on many web sites, and it is SLOW. Its the slowest browser of them all as far as I can tell. It renders many web-2.0′ish sites with teeth-gritting sluggishness. Once rendered, javascript animations tend to be jumpy as well. I’ve found several cases where the event engine is even more screwed up than it was in IE7.
If you’re a web developer and IE8 is causing the same pain for you as it has for me, consider adding this to your HTML*:
* This should be the very first tag after the opening <head> tag.
This will force IE8 to run in IE7 compatibility mode. When in IE7 compatibility mode IE8 will not crash nearly as often and will render your HTML/CSS/JS much more predictably and act more similar to how Chrome and FireFox behave.
I don’t understand why Microsoft keeps doing this. They have all the power in the world to WIN the browser war, but they are just going to keep failing with crap software like this and loose one more market to Google.
Aug/091
Setting up mod_perl6: Part 1
The Thousand Oaks Mongers are going to host a half-day Perl 6 hackathon next Saturday (the 22nd) as already talked about here. So far we have three people involved, including myself. We’d like to create a web site that displays aggregated data from other web sites, such as twitter. We’ve divided our tasks up between the three of us:
- Todd Prest: Database access of some sort. So far Todd has gotten sqlite 3 mostly working and has learned a lot about DBDI (the DBI for perl6) and other bits.
- Shawn Faison: Write some libraries to pull data from various web sites.
- Aran Deltac (me): Get mod_perl6 up and running.
So far my adventures in getting mod_perl6 working have been stalled. Getting rakudo and parrot setup is EASY, just follow these instructions. Once that was setup I went and grabbed mod_parrot. I first grabbed the version 0.05 tarball which failed on make. So, I figured that since 0.05 was released in January its probably way behind the currnet parrot/rakuda development and that I should try out the latest bleeding-edge version of mod_parrot. This time mod_parrot built fine (with lots of warnings) and then failed on make test.
dukeleta over at #parrot valiantly tried to help me resolve the issue, but in the end he figured that the lead developer of mod_parrot, Jeff Horwitz (jhorwitz), would be able to help me out.
The details of my issues are illustrated in this no paste.
I’m not sure how to get ahold of Jeff (he hasn’t been on #parrot for almost a month) and I need to go with the family to the fair.
One way or another, before the hackathon, I’ll either have to find Jeff, figure out how to fix mod_parrot to work, or run an older version of parrot/rakudo that does work with mod_parrot.
Aug/090
use Path::Class;
The CPAN (Comprehensive Perl Archive Network) distribution, Path::Class, provides a simple OO API alternative to the built-in perl file IO functions such as open, opendir, stat, etc. Code written to use Path::Class tends to be smaller (concise, easier to maintain, etc) than similar code that uses the standard built-in functions.
Here is an example of how a small utility might be written without Path::Class:
And here it is re-written to use Path::Class:
Using Path::Class is a no-brainer when you compare `(stat "$dir/$file")[7]` with `$file->stat->size()`.
One of Path::Class’s greatest stengths is that it is very simple and many of the features it provides are done so by glueing with other well-tested modules that have been on CPAN for a long time. These modules include IO::Dir, IO::File, File::Path, and File::stat.
Some other highlights include:
- Has a recurse() method that provides an enhanced File::Find-like interface.
- Works equally well with absolute and relative paths, and can correctly convert between the two.
- Symlinks are handled correctly and there are methods to interact with them.
- Works equally well under different operating systems, such as Linux and Windows.
- The interface, for me anyway, is about as intuitive as it gets. Very well designed to DWIM.
I’ve been using this module at $work for a while now and am very happy with it and have had coworkers tell me, after they gave it a try, how much cleaner their code ended up being because they used this module.
Enjoy!
Jul/092
Upgrading to Catalyst 5.8
Over the last few days at $work I’ve been upgrading Catalyst from 5.7 to 5.8. This new version of catalyst, released in April, has been completely overhauled to use Moose for it’s OO system, among other enhancements and bug fixes.
I found that the Catalyst::Manual::CatalystAndMoose manual was a great starting point to using Moose with Catalyst. Also, Catalyst::Upgrading has specific points to consider when upgrading to 5.8. During the initial install of Catalyst 5.8 I was told that I had a few related modules installed, that while not Catalyst prerequisites, I should upgrade them as they were known to fail on the new Catalyst. I was lucky that I caught this message, as cpan did not pause to wait for me to read it and went on its marry way building, testing, and installing Catalyst.
Once installed I went ahead and also upgraded some related modules we use such as Catalyst::Plugin::Authentication, DBIx::Class, Catalyst::Plugin::Session, Catalyst::Authentication::Store::DBIx::Class, etc. This way I could test all these upgrades all at once instead of one at a time.
Next I modified all my custom plugins to use MRO::Compat instead of Class::C3 per the documentation in Class::C3::Adopt::NEXT. But, then I realized I might as well just rewrite all my plugins as Moose::Roles as its pretty easy to convert a plugin to a role and you end up with much cleaner code.
I started Apache up and things mostly worked. There were a few backwards incompatibilities that I wasn’t aware of, such as there were a few cases where developers had done this:
Instead of:
In our previous version of catalyst (5.7) using :Local, while not documented, worked just like :Private if the subroutine was named index. In 5.8 only :Private will cause a handler named index to be the default handler if the URL resolves to the controller only. This was an easy fix. There were a couple other small issues, but nothing major. I found and fixed a bug in Catalyst::Model::DBIC::Schema and added the support for the compute() method to Catalyst::Plugin::Cache. Both of these changes reduced the amount of changes I needed to make to my own code.
A couple hours in to it I was upgraded and everything was working. Of course I spent the next few days upgrading/enhancing/rewriting Catalyst bits that could be done much more elegantly, and with less code, using Moose.
On a side note, during the upgrade I found this great quote in Catalyst::Model::Adaptor:
Catalyst is glue, not a way of life!
Amen! Most people, myself included, start out by writing all their business logic in their Controllers because that is what MVC told them to do. Don’t do it! Catalyst is meant to be a GLUE – most of your application should be written in a way that it doesn’t give a flying crap how, or by whom, it is used. Put your business logic in modules that optionally expose themselves as Models (perhaps via Catalyst::Model::Adaptor) or just by adding a use statement in your controller that then uses that module. Your controllers should be little more than code to glue a data source with the view and provide some input validation.
I’ve been waiting a long time for this Moosification and its been a joy working with this new Catalyst. Next up to bat will be the Moosification of DBIx::Class! Think we’ll get it by Christmas? :)
Don’t forget to buy the new Catalyst Book! If purchasing from Amazon make sure you follow this link to ensure that the Enlightened Perl Organisation gets a cut of the sale.
Jul/093
XS Hits Newb; Bystanders Watch in Dismay
Last year I wrote GIS::Distance::Fast to provide some much needed speedups to the pure-perl GIS::Distance. This ::Fast version used Inline::C which isn’t all that optimal and was just a scapegoat for doing the right thing.
So, this evening I rolled up my sleeves and went to work. I first looked at a module I already knew to use XS, Cache::Memcached::Fast. That was overwelming.
Then I went on over and read through the perlxs and perlxstut docs. I was STILL floundering about not sure what to do. I then took a whack at using SWIG (Simplified Wrapper and Interface Generator). OhMyGodz I’m feeling overwhelmed now and still nothing works. I’m not going to even document my attempts up until this point.
I then found this document that goes in to more detail about using SWIG with Perl. The thing about this document that helped me so much is that in section 2, Perl Extension Building, it had the simplest example I had yet run in to, and it made sense! Its kinda funny that a SWIG document was what taught me how to write my own XS without using SWIG. :)
An XS distribution can contain as little as two files (well, beyond the usual Makefile.PL, etc). A single .pm file and a .xs file is enough. You can get more complex, if you want, of course. Once you’ve written your Module.xs file you can run it through xssubpp Module.xs > Module.c to see what the .c file looks like and to verify that your .xs file is correct.
Note that xssubpp may complain about you using types that aren’t in your typemap. This is because xssubpp doesn’t know how to find your system’s typemap file. You can specify a typemap file using the -typemap switch. The file is located somewhere in your perl lib path as ExtUtils/typemap.
So, let’s go for an example. Let’s make an XS module that convert hours to minutes. I structure all my code and files as if they would be distributed as a CPAN module. This has added benefits as you’ll see in a moment.
convert-hoursminutes/lib/Convert/HoursMins.pm:
use strict;
use warnings;
our $VERSION = 0.01;
use XSLoader;
XSLoader::load('Convert::HoursMins', $VERSION);
1;
convert-hoursminutes/HoursMins.xs:
#include "perl.h"
#include "XSUB.h"
double convert_hours_mins( double hours ) {
return hours * 60;
}
MODULE = Convert::HoursMins PACKAGE = Convert::HoursMins
PROTOTYPES: DISABLE
double
convert_hours_mins (hours)
double hours
convert-hoursmins/Makefile.PL
name 'Convert-HoursMins';
all_from 'lib/Convert/HoursMins.pm';
requires 'XSLoader';
WriteAll;
convert-hoursmins/t/00_basic.t
use warnings;
use Test::More tests => 2;
BEGIN{ use_ok('Convert::HoursMins'); }
is(
Convert::HoursMins::convert_hours_minutes(1.5),
90,
'1.5 hours is 90 minutes',
);
Now lets see if it worked:
make
make test
Bottom line for me is XS isn’t all that hard when your C code isn’t all that complex. SWIG is overkill for small stuff. But for large swaths of C code I bet SWIG saves lives. Also, XS is a little arcane, but its not too bad.
Jul/090
$(document).ready() Shortcut
When writing jquery you almost always put your code within a ready block:
// Code...
});
And I always forget the exact syntax. I just found that jQuery has a shortcut:
// Code...
});
Much better!
Jul/095
Authoring a CPAN Module
I’ve been authoring CPAN distribution since early 2003 with my first module being Geo::Distance. Since then I’ve authored several other distributions and do my best to keep my RT bug list small and fix FAILs. This process of authoring and maintaining modules on CPAN is in my top-10 list of all-time most educational experiences of my career.
I strongly encourage anyone that wants to become a better developer to take on the task and create a useful distribution for CPAN and/or work on an existing distro. There are plenty of projects out there (such as Moose, DBIx::Class, and Catalyst, to name a few) that need help and can use all the hands that they can get.
Now, if you would instead like to create your own distribution on CPAN, go for it! Its really quite easy and fun. But, it is VERY important that you think long and hard about what you want to release on CPAN. It may seem that every possible task that someone might want to accomplish in Perl is already available on CPAN – this is false. I regularly find myself needing tools that aren’t yet on CPAN. Don’t just write something for the sake of getting something on CPAN – develop something that is useful, of high quality, has a decent suite of automated tests, and doesn’t duplicate a module that is already on CPAN.
Once you developed a module or two, and you are ready to get it out the door, you’ll want to do a couple things. First, create a directory to hold your distribution. Assuming your module is named Acme::Yesterday:
cd Acme-Yesterday
vi Makefile.PL
You’ll want to install Module::Install first, before you go any further as it provides a much easier to configure build/test/install process than the old-school ExtUtils::MakeMaker.
Now, your Makefile.PL will look something like this:
all_from lib/Acme/Yesterday.pm
requires DateTime
test_requires Test::More 0.42
(Notice the first line DOES have a semicolon at the end, while the rest do not)
Next, create a directory to put your module(s):
mkdir lib/Acme
vi lib/Acme/Yesterday.pm
Now, your module will need some minimum requirements such as declaring the version of the distribution, the license type, and a name. Also, most CPAN modules follow a common convention for documentation using POD. Here is the most minimal of a module:
use strict;
use warnings;
our $VERSION = 0.01;
=head1 NAME
Acme::Yesterday - Make time() return the same hour,
second, and even minute as exactly 24 hours ago!
=head1 SYNOPSIS
use Acme::Yesterday;
=head1 DESCRIPTION
I'm an interesting introduction to this module.
=cut
use DateTime;
=head1 METHODS
=head2 some_method
some_method();
Description of some_method().
=cut
sub some_method { ... }
1;
=head1 AUTHOR
Your Name <your@email>
=head1 COPYRIGHT
This program is free software; you can redistribute it
and/or modify it under the same terms as Perl itself.
Alternatively you can use Module::Starter to create a shell of a module for you.
Now for automated tests. At a minimum you should write a test that verifies that your module can even be loaded. Here’s how to do that:
vi t/00_use.t
The content of 00_use.t should be something like:
use strict;
use warnings;
use Test::More tests => 1;
BEGIN{ use_ok('Acme::Yesterday'); }
You’ll need to make a couple more files:
echo '0.01 - First release.' > Changes
OK, now just sit back and let Module::Install do its work:
make dist
make disttest
Assuming your tests pass when you run `make disttest`, you’re set to put your module on CPAN. The above steps would have now created for you a Acme-Yesterday-0.01.tar.gz. This is the file that you need to upload to CPAN. In order to upload a distribution to CPAN you’ll need to get a pause account. Once your request has been approved you can login to pause, go to the upload page, and upload your tarball. Within an hour or two your new distribution will be availabe on CPAN.
Once you’ve done this you’ll want to clean up your distribution directory as running the various make steps leaves around some files you don’t need:
Good luck!
