All ‘Programming’ Posts

Looking Back at Old Code

No Gravatar

I just came across the following gem in some PHP code I wrote for a web application about seven years ago:

for ($i = 0; $i < count($items); $i++) $items[$i] = $items[$i];

I cannot begin to understand the logic behind this. I've tried to plumb the depths of my knowledge of PHP and how variables can cause E_NOTICE messages when they are referenced while uninitialized, but I simply can't figure out what I was trying to accomplish with that line of code.

It makes me happy to look back on what I have done and realize how much I've learned since then.

Why People Don’t Like Objective-C

No Gravatar

I’m sure it’s unfair to call this a representational example, but it’s funny, so I’m sharing it. I haven’t actually used Objective-C before, so please don’t take this as anything more than a joke.

This is a segment of example code that Toodledo has in their API documentation:

Generating the signature with PHP

$key = md5( $userid.$myAppToken );

Generating the signature with Obj-C

char *cStr =
    [[NSString stringWithFormat:@"%@%@",userid,myAppToken] UTF8String];
unsigned char result[CC_MD5_DIGEST_LENGTH];
CC_MD5(cStr, strlen(cStr), result);
NSString *sig = [NSString stringWithFormat:
    @"%02X%02X%02X%02X%02X%02X%02X%02X%02X%02X%02X%02X%02X%02X%02X%02X",
    result[0], result[1], result[2], result[3], result[4], result[5],
    result[6], result[7], result[8], result[9], result[10], result[11],
    result[12], result[13], result[14], result[15]];

It kind of speaks for itself, doesn’t it?

PDOQuery: jQuery Style Database Access and Manipulation

PDOQuery Logo
No Gravatar

I’ve been working on a new database library recently. It’s written in PHP, since that’s what I need, but nothing says it couldn’t be ported to other languages (possibly Java, Python, or Ruby). I’ve had a de-facto library that I’ve used for my projects for nearly six years, but it just isn’t cutting it anymore.

My old one is a relatively simple generic class with standard CRUD functions for batch and single primary-key-based operations. I wrote it myself when I first learned about object-oriented programming, and I’ve made a few tweaks here and there through the years. It’s just too heavy though, and not very intuitive. Anyone unfamiliar with my code would really have to study the library for a long time to figure out how to use certain parts—I am sure of this because even I have to really study it sometimes to refresh my memory. That’s a bad sign when the library in question is a simple database access layer.

Enter PDOQuery. PDOQuery is my new library. It’s modeled to a limited extent after jQuery syntax, though obviously not all the same kinds of functionality apply, and the language difference (PHP vs. Javascript) means that PDOQuery uses -> for chaining instead of a period. The query language is built in a similar fashion, only PDOQuery selects and manipulates a database instead of DOM elements.

It’s also meant to give you as much power as possible with as little pain as possible. Query modifications are built to return a PDOQuery object, just like jQuery, so you can link modifications together or apply a new selector to an existing object. The library takes full advantage of PHP to provide powerful object integration, making it extremely simple (and mostly automatic) to create a PHP class that directly represents a database table row. It also supports table references, so with a single line of PHP, you can select a user and all related tasks, for example.

Consider the following implementation code:

include 'PDOQuery.class.php';
PQ() -> connect("mysqli:dbname=testdatabase", "testuser", "testpass");

The PQ() function is a shortcut to the main PDOQuery class. It takes an optional selector argument and returns an instance of the class. Note that this single include and call to the connect() method are the only bits of code necessary to start using the library. Now for some usage:

$s = PQ("users.join:permissions[UserID]");
$users = $s -> select();

The first line above creates a PDOQuery object that parses the selector into appropriate MySQL query, but it doesn’t actually run the query. The second line is what runs it and stores the result. For the MySQL database type (the only one coded at this point), the default result returned is a native MySQL result. You can optionally request a numerically indexed array of rows, an associative array of rows, a numerically and associatively indexed array of rows, or an array of objects. The native result is the default because it has the smallest memory requirement.

You could also run the following code:

$s -> delete();

…though I wouldn’t recommend it. Because the selector in the $s object is parsed and stored in individual pieces (table name, criteria, joins, etc.), it doesn’t assume you’re going to just be selecting. The line above would use all of the selector parts applicable to a DELETE query (just the table name, in this case), ignoring all the rest (the JOIN option), and run the following MySQL query:

DELETE FROM `users`

You could also do the following:

$s2 = $s -> where("UserID=3");
$s2 -> delete();

This would delete only the user whose UserID is equal to 3. Much safer.

Currently, the PDOQuery library supports the following database-related methods:

  • query()
  • select()
  • selectRow()
  • selectColumn()
  • selectValue()
  • insert()
  • update()
  • delete()

The query() method is used by all the others, and can be used by you to run a raw query if you cannot accomplish it using other methods. It returns a native database result. The selectRow() method returns a single row, selectColumn() returns an array of values from a single column and multiple rows, and selectValue() returns the value of one column’s content from within a single row.

PDOQuery also has these chainable selection methods:

  • join()
  • naturaljoin()
  • leftjoin()
  • rightjoin()
  • outerjoin()
  • leftouterjoin()
  • rightouterjoin()
  • where()
  • groupby()
  • orderby()
  • having()
  • limit()

The outerjoin() method is a the exact same as leftouterjoin(). All of these return a PDOQuery object for further chaining or manipulation.

However, while this stuff is all important, it isn’t the best feature of the library. PDOQuery includes a discover() method which creates a map of fields, data types, and indexes of tables in the database. This information can then be used to generate code for you to inject into your object classes to enable many convenient shortcuts. You can add just a bit more information yourself to establish relationships between tables for even greater conveniences.

Consider a database with a users table and a projects table. A user may have zero or more projects. The following code is mostly generated by PDOQuery, except for the class names and the references index keys:

class User extends PDOQueryObject {

    public static $PDOQueryTable = "users";
    public static $PDOQueryStructure = array(
        "timestamp"=>"Fri, 25 Jun 2010 10:27:02 -0600",
        "fields"=>array(
            "UserId"=>array("type"=>"int(10) unsigned", "simpletype"=>"int", "null"=>false, "key"=>"PRI", "default"=>null, "ai"=>true, "indexes"=>array("PRIMARY"), "extra"=>"auto_increment"),
            "EmailAddress"=>array("type"=>"varchar(128)", "simpletype"=>"text", "null"=>false, "key"=>"", "default"=>null, "ai"=>false, "indexes"=>array(), "extra"=>""),
            "Password"=>array("type"=>"varchar(32)", "simpletype"=>"text", "null"=>false, "key"=>"", "default"=>null, "ai"=>false, "indexes"=>array(), "extra"=>""),
            "FirstName"=>array("type"=>"varchar(32)", "simpletype"=>"text", "null"=>false, "key"=>"", "default"=>null, "ai"=>false, "indexes"=>array(), "extra"=>""),
            "LastName"=>array("type"=>"varchar(32)", "simpletype"=>"text", "null"=>false, "key"=>"", "default"=>null, "ai"=>false, "indexes"=>array(), "extra"=>""),
            "LastLogin"=>array("type"=>"datetime", "simpletype"=>"datetime", "null"=>false, "key"=>"", "default"=>null, "ai"=>false, "indexes"=>array(), "extra"=>""),
            "Deleted"=>array("type"=>"tinyint(4)", "simpletype"=>"int", "null"=>false, "key"=>"", "default"=>null, "ai"=>false, "indexes"=>array(), "extra"=>"")
            ),
        "indexes"=>array(
            "PRIMARY"=>array("unique"=>"", "collation"=>"A", "cardinality"=>"1", "fields"=>array("UserId")),
            ),
        "primarykey"=>"UserId",
        /* USER-SUPPLIED SETTINGS */
        "references"=>array(
            "projects"=>array("key"=>"UserId", "autojoin"=>false, "relationship"=>"multiple", "class"=>"Project", "local"=>"Projects"),
            )
        );
}

class Project extends PDOQueryObject {

    public static $PDOQueryTable = "projects";
    public static $PDOQueryStructure = array(
        "timestamp"=>"Fri, 25 Jun 2010 10:27:02 -0600",
        "fields"=>array(
            "ProjectId"=>array("type"=>"int(10) unsigned", "simpletype"=>"int", "null"=>false, "key"=>"PRI", "default"=>null, "ai"=>true, "indexes"=>array("PRIMARY"), "extra"=>"auto_increment"),
            "UserId"=>array("type"=>"int(10) unsigned", "simpletype"=>"int", "null"=>false, "key"=>"", "default"=>null, "ai"=>false, "indexes"=>array(), "extra"=>""),
            "Name"=>array("type"=>"varchar(64)", "simpletype"=>"text", "null"=>false, "key"=>"", "default"=>null, "ai"=>false, "indexes"=>array(), "extra"=>""),
            "Deleted"=>array("type"=>"tinyint(4)", "simpletype"=>"int", "null"=>false, "key"=>"", "default"=>null, "ai"=>false, "indexes"=>array(), "extra"=>"")
            ),
        "indexes"=>array(
            "PRIMARY"=>array("unique"=>true, "collation"=>"A", "cardinality"=>"0", "fields"=>array("ProjectId"))
            ),
        "primarykey"=>"ProjectId",
        /* USER-SUPPLIED SETTINGS */
        "references"=>array(
            "users"=>array("key"=>"UserId", "autojoin"=>true, "relationship"=>"single", "class"=>"User", "local"=>"User")
            )
        );
}

Notice the extended PDOQueryObject class, which provides most of the convenience functions. Also notice the references key down at the bottom of each class definition. For each of these:

  • key is the correlating field name
  • autojoin determines whether to select this related information automatically
  • relationship is either “single” or “multiple” depending on whether it is 1-to-1 or 1-to-many
  • class is the name of the class that represents the joined table rows
  • local is the name of the local variable to assign the joined data

This allows us to do something like this:

$user = new User(1);

Now, $user is an object that contains a copy of the data row from the users table that has UserID equal to 1. The fields are stored in respective member variables: UserID, EmailAddress, Password, etc. These can be modified directly, since they are public. If you want to make changes, simple assign new values to the fields, then run the following code:

$user -> update();

Voila! Because that update() method is running on an instance of a single User class, it knows to treat it as a database row, and it already has all of the table structure and primary key information handy. It even performs automatic data validation and formatting (most conveniently for DATE, TIME, and DATETIME fields). You could even assign a new UserID value (or set it to NULL) to your $user object and then run:

$newID = $user -> insert();

…and you’ve got a new row inserted, just like that. But wait, there’s more! Because of the references array value, we can also do this:

$user -> loadReference("projects");

…and now you have a new Projects member variable which contains an array of objects pulled from the database and instantiated using the PDOQueryObject child Project class. If the autojoin value had been set to true, this would have been done for us automatically. Now that is convenient.

I’m still working on a few more convenience functions (data table and entry form displays, and lambda functions for automating per-row operations), and I still have to finish the documentation. It’s going to be really, really useful at least for me, and hopefully for many others. I’ll release the code as soon as I’m satisfied with it.

The Importance of Multiple Options

No Gravatar

I recently finished up an article summarizing the differences between a few of the mobile application development frameworks that are in existence today. I covered three major players, two individual platform-specific tools, one per-device shortcut approach, and my own fledgling project, AML. After researching and writing, here’s what I concluded:

  1. There are a lot of people trying to accomplish the same goal in different ways.
  2. That’s really great.

Now, honestly, the first point was already assumed. Anyone who’s done a simple web search on the topic would realize that. But what about the second point? More directly relevant in my case, what about my AML project, which is entering a field that already has a lot of competition and projects which have a great big head start over mine? Why would it be great for my project to have a lot of competition?

Well, first of all, I didn’t look at the playing field and think, “How can I do what they are doing, only better?” No, I started with a specific personal need. I have a problem that I need to solve, and I thought up what I believe is a great way to solve it. Then I searched for something that did what I needed, and came up empty. There were some things that were similar, of course, but nothing that did exactly what I wanted. So, I decided to build my own tool. Even if it ends up being similar to the other ones, I am still going to build it because it will solve my problem perfectly, and there are probably other developers out there who will need to solve the same kind of problem who will benefit from my project.

That is why having all of those different options is so great. If I were trying to solve a slightly different problem, I could use an existing tool, because somebody else already did the same thing I did, only they had a different problem to solve. Instead of everyone needing to bend a universal tool to fit their criteria, we have a multitude of choices, each with its own strengths and weaknesses.

In the case of mobile platform development frameworks, this is excellent. I wouldn’t want to have only one of them to choose from, since I can easily come up with a problem for each framework that would be a pain to solve completely if that was the only one available. Everyone has their own idea of what is best, and this creates a competitive arena. They each build what they need or want, taking inspiration from other designs, and come up with a solution. Sometimes the products converge into a single product, and sometimes they remain separate, but they continue to give developers different options to choose from.

This is really just the principle of the free market applied to software development, which explains why I appreciate it so much.

Rapid Android Development with AML

No Gravatar

7/13/2010 update:

Thanks to the great response (and many hundreds of visits to this page over the last two days), I’ve gone ahead and created a real project website for AML at http://www.amlcode.com, as well as the @amlcode Twitter account. If you’re interested in more updates, check those out. Thanks for all the comments and ideas! Keep them coming!

This post still has some good info and some great comments, so don’t neglect it completely. It might get lonely.

I gave myself a crash course in Android development over the last few days. I’d officially been part of an Android team as part of a project management class in my last quarter at Cal Poly Pomona, but some of the other members on the team ended up doing most of the actual development, so my instructive coding experience was pretty minimal. I knew enough to get the Android SDK installed in Eclipse (which is painfully slow, by the way!), but that was all. Everything else was new to me.

Now, I am very familiar with C, C++, and Java, even though most of my coding for my job is done in PHP, so picking up the Android vocabulary within Java was much easier for me than it would have been if I had no programming experience at all.

I don’t have a specifically defined project to work on yet, but I do have some general requirements for the Android app(s) I’m sure I will be developing in the near future. Specifically, I know I will need these features:

  1. Native implementation (Android UI, not HTML5)
  2. Dynamic externally generated content and navigation

Both of those two features together are, I believe, not a common combination. It’s simple to have dynamic externally generated content and navigation, if you use HTML and a browser. It’s also simple to have a native UI support externally generated content, if that content always fits into a pre-defined structure which you build into your app. But what if you want an app that supports a system with very little known ahead of time—just an email address and password, for instance—and the rest of it need is determined based on parameters, permissions, and a web service API?

Then, I thought, why not write a native Android library that will accommodate that very thing?

And that’s what I did.

I even had the presumptuous audacity to give it an official-sounding name, though it is by no means official (or truly useful) just yet. There is no W3C specification for it. And for all I care, it may never become official, but even if I’m the only one who uses it, this library it will certainly make things easier for me. And on top of that, it’s been a great learning experience over the last four days.

Ladies and gentlemen, I introduce to you AML, the App Markup Language, a simple XML-based language that allows you to easily build a clean, functional application for your mobile device.

It resembles HTML in some ways, but it is not a subset of HTML. It is designed to be intuitive and easy, and while it does not implement every possible feature achievable with native Android code, the goal is for it to cover the vast majority of what is necessary for information-based apps, and leave the rest up to you to implement as you desire. It probably won’t help out much with games, but it’s perfect for mobile implementations of websites. It can even be used as part of a local-only application to quickly build the UI instead of using Java code or Android’s layout XML structure.

Additionally, I would love at some point to extend AML in two directions: one, to have it support other devices (namely the iPhone and iPad), and two, to support different languages for the markup (namely JSON, perhaps YAML). While I don’t need these right now and may not actually need them for some time, there is nothing really Android-specific or XML-specific about the language. I have never written code for iOS, so I don’t know if the behavior is inherently different enough to cause a problem, but I bet it isn’t. You should be able to use the same AML markup to generate nearly identical experiences on Android and on iOS.

So, why is AML so cool? Why would anyone want to use it? Check this out.

The images on the right shows a set of different input objects in an Android app. It’s just an example, obviously—an app in the Market that looked like this would be odd indeed. However, it does illustrate a very basic layout with a few different objects.

Now, anyone who has done any Android development know that there isn’t anything truly difficult about building something like this. If you write your own “main.xml” layout file and use it for your Activity object’s content view though, it can be a little tedious. You can use a tool like DroidDraw to speed things up (which is an excellent idea if you need a static layout), but with both of these approaches, your design will be hard-coded into your app. You can only make changes on the fly through clever uses of the LayoutInflater service.

Your other option is to build the layout using only code. This allows for more dynamic design, for sure, but doing this manually is still tedious. AML bridges the gap between an external data source and the native application, so that the external source can very simply instruct the app what to build and how it should behave.

This is the Android layout XML necessary to create the layout shown above in the screenshot on the right:

<LinearLayout
    android:orientation="vertical"
    android:layout_width="fill_parent"
    android:layout_height="fill_parent">
    <Button
        android:layout_width="wrap_content"
        android:layout_height="wrap_content"
        android:text="Test Button" />
    <CheckBox
        android:layout_width="wrap_content"
        android:layout_height="wrap_content"
        android:text="Test Checkbox" />
    <ImageButton
        android:layout_width="wrap_content"
        android:layout_height="wrap_content"
        android:src="@drawable/icon" />
    <ToggleButton
        android:layout_width="wrap_content"
        android:layout_height="wrap_content"
        android:checked="true" />
    <EditText
        android:layout_width="wrap_content"
        android:layout_height="wrap_content"
        android:text="Test Text"
        android:singleLine="true" />
    <EditText
        android:layout_width="wrap_content"
        android:layout_height="wrap_content"
        android:text="Test Password"
        android:password="true"
        android:singleLine="true" />
    <RadioGroup
        android:layout_width="fill_parent"
        android:layout_height="wrap_content"
        android:orientation="vertical">
        <RadioButton
            android:layout_width="wrap_content"
            android:layout_height="wrap_content"
            android:checked="true"
            android:text="Radio Button 1" />
        <RadioButton
            android:layout_width="wrap_content"
            android:layout_height="wrap_content"
            android:text="Radio Button 2" />
        <RadioButton
            android:layout_width="wrap_content"
            android:layout_height="wrap_content"
            android:text="Radio Button 3" />
    </RadioGroup>
</LinearLayout>

Keep in mind that the above code is basically fixed, and cannot really be customized at runtime on a large scale. Now, for comparison, here is the AML necessary to create the layout above:

<aml>
  <input type="button">Test Button</input>
  <input type="checkbox">Test CheckBox</input>
  <input type="togglebutton" checked="yes" />
  <input type="imagebutton" image="drawable/icon" />
  <input type="text">Test Text</input>
  <input type="password">Test Password</input>
  <radiogroup>
    <input type="radiobutton" checked="yes">Radio Button 1</input>
    <input type="radiobutton">Radio Button 2</input>
    <input type="radiobutton">Radio Button 3</input>
  </radiogroup>
</aml>

It’s that simple. Additionally, for my tests, this layout info was actually being pulled from a web service, just so I could prove to myself that I could do it. The web service dynamically built that layout.

Here is a sample Android ListView constructed with the following AML:

<aml>
  <list>
    <item>Sample Test List Item A</item>
    <item>Sample Test List Item B</item>
    <item>Sample Test List Item C</item>
    <item>Sample Test List Item D</item>
    <item>Sample Test List Item E</item>
    <item>Sample Test List Item F</item>
    <item>Sample Test List Item G</item>
    <item>Sample Test List Item H</item>
    <item>Sample Test List Item I</item>
    <item>Sample Test List Item J</item>
    <item>Sample Test List Item K</item>
    <item>Sample Test List Item L</item>
    <item>Sample Test List Item M</item>
    <item>Sample Test List Item N</item>
    <item>Sample Test List Item O</item>
    <item>Sample Test List Item P</item>
    <item>Sample Test List Item Q</item>
    <item>Sample Test List Item R</item>
    <item>Sample Test List Item S</item>
    <item>Sample Test List Item T</item>
    <item>Sample Test List Item U</item>
    <item>Sample Test List Item V</item>
    <item>Sample Test List Item W</item>
    <item>Sample Test List Item X</item>
    <item>Sample Test List Item Y</item>
    <item>Sample Test List Item Z</item>
  </list>
</aml>
And another sample TableLayout that uses alignment and column spanning, generated by the following AML:

<aml>
  <table expand="2">
    <tr>
      <td colspan="2">This is a block of text contained within a single table cell. This cell has a colspan of 2 so that it looks good on top of the label/field pairs below.</td>
    </tr>
    <tr>
      <td align="right">Email:</td>
      <td><input type="text" width="fill" /></td>
    </tr>
    <tr>
      <td align="right">Password:</td>
      <td><input type="password" width="fill" /></td>
    </tr>
    <tr>
      <td colspan="2">Here's another 2-column cell. Is this awesome or what?</td>
    </tr>
    <tr>
      <td colspan="2">Notice the slight visual break from the text above. This text is in yet another row. The field labels above on the left are aligned to the right, and the table is instructed to make the 2nd column expandable (like an "auto-width" kind of thing).</td>
    </tr>
    <tr>
      <td colspan="2">Additionally, the fields have their widths set to "auto" so that even though they are empty, they fill their parent cells.</td>
    </tr>
  </table>
</aml>

Seriously, can it get any easier than that? You can store this kind of layout definition in your app code and reference it directly from there, or you can use a web service to build this stuff on the fly and send it to your app (though you might want to implement some sort of caching mechanism in that case, for efficiency). The AML library is in its own package, and doesn’t require any imports from your package to work. You do need to put the template layout XML files into your /res/layout folder, but that’s all. AML doesn’t require you or your web service to know the details of how Android builds its views. It doesn’t depend on some extra plugin installed on your device, and it isn’t passing your app code through another service. You probably still have to write some Java code to accomplish your goal, but this makes the whole thing easier.

So far, what I have written only supports layouts, but no actions. Many objects support attributes like align, valign, fontsize, padding, color, and bgcolor, to name a few. The rest of the code just isn’t there yet. I have been building the tap and hold attributes into buttons and other clickable objects, but I’m not done. I haven’t done a date picker, time picker, or spinner (Android’s version of an HTML <select> element). But it’s coming together very quickly—a lot faster than I thought it might. It’s been an awesome learning experience so far though.

I’m writing this for me, honestly, but I’m really interested in any kind of outside interest. Any questions, comments, or recommendations? Has anyone else done this? Would you be excited to get your hands on this code? If you used it, what would you need it to be able to do?

Let me know in the comments!

Copy Protection Fun with Assembly and Machine Code

No Gravatar

I recently had the enjoyably challenging opportunity to patch a very, very old bit of software to defeat some very basic copy protection. (DISCLAIMER: this post doesn’t describe how to circumvent any kind of new application or game protection, which is typically orders of magnitude more complicated than this old stuff.)

[UPDATE 6/17/2010: see links at end of post for assembly code references]

Twice now, I’ve found myself with an ancient, unsupported program that has a pointless copy protection mechanism in place. This particular one required two codes to be entered into a separate activation program, which would then create a separate hidden file that the main program would read to detect proper activation. Fortunately, I learned just enough in my Assembly class from Fullerton College that looking at a disassembled DOS executable doesn’t completely intimidate me. I don’t know most of it, but I do understand basic instructions like mov, jmp, jz, and the use of registers and flags.

What’s nice about old software is that the activation code can usually be traced down to a single logical test, which is just a logical operation followed by a conditional jump. The trick is to find this code, and then replace the conditional jump with either a no-op or an unconditional jump, depending on whether the original code jumps to a failure or success location. I have used this technique to write a no-CD patch, and now also an activation bypass patch. You need a couple of tools like IDA Pro (not free) or PVDasm (free) for disassembly, and then another one like HxD (free) for straightforward hex editing.

Here’s some clipped disassembled code from this last project:

seg008:12D3 call sub_1967F
seg008:12D6 or al, al
seg008:12D8 jz short loc_1A7FB

seg008:1309 jmp short loc_1A808
. . .
seg008:130B ; ----------------------------------------------------------------
seg008:130B
seg008:130B loc_1A7FB: ; CODE XREF: sub_1A75B+6D j
seg008:130B mov al, 0
seg008:130D push ax
seg008:130E mov di, offset aActiveError ; "Activation Error."...
seg008:1311 push cs
seg008:1312 push di
seg008:1313 call sub_2C89A
seg008:1318
seg008:1318 loc_1A808: ; CODE XREF: sub_1A75B+9E j
seg008:1318 mov sp, bp
seg008:131A pop bp
seg008:131B retf
seg008:131B sub_1A75B endp

The important bit is the bold red at “seg008:12D6″ and “seg008:12D8″. The “or al, al” is the logical instruction which sets the “zero” flag if the “al” register is equal to zero. Then the “jz” instruction right after it is a conditional jump to that “loc_1A7FB” label shown below. That’s the part that displays the activation failure message, so I figured that conditional jump was the code to change. In machine code, it’s just two bytes: 0×74 (the jz opcode) and a 1-byte memory offset that points to the new location. I just replaced those two bytes in the .exe file with 0×90 (no-op), and voila! Now it never, ever jumps to the error message, and instead always just continues.

Chalk another one up to machine language succeeding where high-level languages are unusable.

UPDATE 6/17/2010: I’ve found a couple of excellent reference guides that help when you need to make direct edits to machine code:

X86 Opcode and Instruction Reference Home (byte-to-instruction reference)
http://ref.x86asm.net/coder32.html

DOS INT 21h – DOS Function Codes (disk access and some I/O reference)
http://spike.scu.edu.au/~barry/interrupts.html

Using SQL to Find Unscheduled Hours

SQL Appointment Database
No Gravatar

I recently needed to come up with a good way to find available slots of time in a schedule for a system that could give the user the opportunity to choose their preferred time slot. The appointment list is stored in MySQL as a set of entries which contain, among other irrelevant things, a start date/time and end date/time. I would guess that pretty much any scheduling system built in a relational database would have a basic structure similar to this.

Now, usually, SQL database queries must come up with clever ways to locate and sort records that are in the database somewhere. I had the interesting task of using SQL to show me only what was not in the database. The appointments table doesn’t store records by hour (or any other segment of time), it stores records by appointment only. A couple of years ago, I created some code that accomplished the basic goal, but it relied heavily on PHP post-processing of the selected appointment records, and it was very, very slow. What I came up with now is significantly simpler, much faster, and elegant enough that I felt like posting my own code online (which doesn’t really happen that often).

I’m not sure if this works in other SQL engines besides MySQL, but I would guess that it does. It may depend to some degree on the order in which query variables are evaluated and subqueries are executed, but I don’t know if there is any difference in this behavior between different engines, or if it’s part of the SQL specification. This query is built assuming your table is named appointments, and it contains AppointmentId, StartTime, and EndTime columns. This is also using an arbitrary search window of 5/15/2010 10:00am through 5/30/2010 10:00am, and a minimum gap length of 60 minutes.


SELECT AFrom.FromEndTime                               AS `StartTime`,
       TIMESTAMPDIFF(MINUTE, FromEndTime, ToStartTime) AS `Length`
FROM   (SELECT @rownum := 0 AS                            Reset1
       ) AS TReset1,
       (SELECT DISTINCT appointments.AppointmentId AS FromAppointmentId,
                        @rownum := @rownum + 1 AS FromNum,
                        appointments.EndTime       AS FromEndTime
       FROM             appointments
       WHERE            StartTime BETWEEN
                            "2010-05-15 10:00:00" AND
                            "2010-05-30 10:00:00"
       ORDER BY         StartTime
       ) AS AFrom,
       (SELECT @rownum := 0 AS Reset2
       ) AS TReset2,
       (SELECT DISTINCT appointments.AppointmentId AS ToAppointmentId,
                        @rownum := @rownum + 1 AS ToNum,
                        appointments.StartTime     AS ToStartTime
       FROM             appointments
       WHERE            StartTime BETWEEN
                            "2010-05-15 10:00:00" AND
                            "2010-05-30 10:00:00"
       ORDER BY         StartTime
       ) AS ATo
WHERE  AFrom.FromNum = ATo.ToNum - 1
AND    TIMESTAMPDIFF(MINUTE, FromEndTime, ToStartTime) >= 60



This returns a set of records with two fields: StartTime, which contains the start of the gap, and Length which is the length in minutes. Changing the unit of time is a simple as changing the MINUTE in the TIMESTAMPDIFF function to be some other MySQL-friendly unit. I needed minutes, so that’s what I used.

The beauty in this approach is that it uses a rudimentary self-join (WHERE AFrom.FromNum = ATo.ToNum - 1) based on the SQL-generated per-row counter variable (@rownum). It is effectively joining each row with the one after it (and ignoring the very last one). I didn’t even know that was possible until I got this query to work. It uses two otherwise ignored subqueries to reset the counter variable to zero before executing the next subquery, which solves the problems of initializing the counter and of the counter continuing to increment after the end of the first subquery of appointments.

This approach has a couple of limitations:

  1. The very last appointment is ignored because of the way the self-join works. It can’t be joined to the next appointment because there is no next appointment, and so gets filtered out. This isn’t a problem for me because I can simply extend the window of time farther out if I need more appointments, but it’s something to consider.
  2. If there is a gap between the beginning of the window and the first appointment, it is ignored. This is because the query basically gives you only the gaps between appointments, which means if the first appointment starts after the beginning of the specified window, then that gap isn’t noticed. Something must be present in your code to accommodate for this.

Those are the only issues that have come up for me so far. I hope I don’t find anymore; this thing has been working beautifully as it is now. It’s orders of magnitude better than my last implementation. I’m interested in any bug reports or other limitations though.

PHP Object Programming Paradigm Shift

No Gravatar

I’ve recently started trying to implement a better way to structure my code. It’s almost as big of a shift as when I switched from procedural to object-oriented programming. The funny thing is that it isn’t a new approach at all. It’s just the true application of OOP. I’ve been doing it for years in assignments for the various Java and C++ classes I’ve taken at Fullerton and Cal Poly. It’s normal and intuitive. Each class represents one data structure. Duh.

Only, that’s not how I’ve been writing my PHP code. Not exactly, anyway. My PHP has been written in a more “database-oriented” style, rather than an object-oriented style. All of my classes were built around database operations—usually dealing with sets of data rows—rather than individual objects. If you’ve ever written a class for a data structure, you know that one of the most common features is a set of member variables that correspond to the object properties. My classes never had those, mainly because PHP’s associative array structure can represent objects very nicely.

This approach leads to only a partial representation of the object. I would create only a single instance of each class, and use that to interface with the database and perform all of the operations. You can get by this way with PHP, because it’s a language that doesn’t (usually) create interactive programs that reside in memory; each time a script runs, it’s a single hit to a webpage rather than a persistent instance. But it still leaves things a bit convoluted from a design standpoint. At least, that’s what I’m beginning to discover. Classes need to define objects and their relevant properties and operations, nothing else.

So, I’m working on a library that will help separate database stuff from object stuff, so that objects can be objects and database operations happen separately, but the database aspect of things can be attached cleanly and simply no matter what the objects look like. Hopefully I’ll have some code to give away soon, once I’m happy with it.

I love learning new things.

The Evolution of a Programmer

No Gravatar

I think I am a pretty good programmer. I only feel free to say this because I have had many other people (including other programmers) tell me so over the last many years. I know that there are many programmers out there who are better than I am, and not just hypothetically—I’ve met them in college classes and online. However, it really doesn’t matter how good I am for the sake of this argument. What matters is that I think I am good. And even more importantly, a year ago, I felt exactly the same way about myself. Looking back, I know that I was comparatively lacking in many areas.

This can be a discouraging revelation. Every time I have to work with code that I wrote a year or more ago, it’s all I can do not to bang my head against my keyboard. I always find myself wanting to rewrite the whole function (or class, or website) because I know I could just do so much better now than what I did before. I want to use some new coding model, a new CSS convention, a more efficient database structure, or any other simpler/better/faster technique to make an old convoluted mess into a thing of beauty. Making mere modifications to such blatantly awful code is painful. I feel almost ashamed, even if just a little bit, even if only for a moment inside, to have written code that is sub-par at best.

Recently though, I’ve come to interpret that revelation as an encouragement.

Computer technology is one of the fastest-changing fields around. And I don’t just mean the hardware, which is typically the subject of that sort of statement. The same thing applies to software applications and, by implication, the programming languages used to write them. This is most severely felt in the web application market, where new languages and frameworks seem to come into existence weekly. If you are in the mood to learn something new, there are endless opportunities, many of which could probably benefit your specific job directly in some way.

This is even true if you stay mainly within one language. I’ve been programming almost exclusively in PHP since about 2003, but during the last six years, PHP has undergone many revisions and updates (most notably, a true object-oriented implementation instead of a hack-job imitation). I am constantly learning new tricks, and occasionally learning totally new ideas that change the way I use the language. I wrote my own MVC framework about a year ago, and I learned so much new stuff about PHP during the process that my whole programming paradigm changed. I couldn’t believe that I’d gone so long before learning what seemed so fundamental. That kind of thing doesn’t happen all the time to me, but I do learn how to do my job better on a regular basis.

It’s this learning process that enables me to recognize how bad my old code was, and this is what is so encouraging. If I never realized my old code was bad, it would mean that I hadn’t learned anything new, and that would be a truly disheartening outcome. I would much rather be constantly reminded of how much farther I have progressed than be unknowingly stagnant in a rapidly changing field.

Of course, it is possible that you’re just totally awesome at what you do, and your code is impossible to improve. If that’s the case, please email me so we can set up a brainstorming session. Or perhaps you’re using an ancient language (COBOL or FORTRAN, anyone?) because your job requires it, and you can’t really learn new things because the last language revision was decades ago. But for most web developers, neither of those things are true.

So, even if you don’t need to delve into your old code to fix or modify anything, take a moment sometime and compare what you’ve done in the past to what you do now. Bash your head on the keyboard if necessary; but if you do, realize you’re only inclined to do so because of how much farther you’ve come.

More school and programming fun

No Gravatar

Okay, so it's been almost three and a half months since my last post. So much for maintaining a decent frequency. At least I've been twittering more often than that. And no, I can't say that it's really been too much more interesting there than the silence you've all (both?) seen here.

The last eight weeks have gone by pretty quickly. That's a good thing, because I'm eager for the summer for a few reasons. This quarter has been filled with core theoretical computer science classes, all of which are interesting, and some of which are harder than others. I have symbolic programming (LISP and Prolog), language translation and automata (focusing on ideas behind programming language parsing and structure), and computer architecture (focusing on MIPS). It sounds a lot more complicated that it is, at least if you've been going through the classes from the beginning. I had very little idea about any of this stuff eight weeks ago. I'm doing pretty well with it so far though. Only 16 days until the quarter ends! I'm on track for Spring 2009 graduation, which has been my target for quite some time.

Some of the most interesting school-related stuff that's happened has been with the LISP and Prolog class I'm taking this quarter. I'd say it's definitely my favorite out of the three I'm taking right now, because it's a new language, and it's just really interesting. LISP is like Scheme, which probably doesn't mean anything to most of you. It's unlike procedural languages (BASIC, C, etc.), and unlike object-oriented languages (C++, Java, etc.). It's what they call “functional.” Everything you write is a function, literally. It takes some getting used to, but you can do some really handy things with it a lot more easily than you can with other languages. Prolog, which we just started going over last week, is even more radically different. It's a “logical” language, where you first define things that are true, and then you basically ask yes/no questions about different things, and Prolog tells you the answer based on what it knows. It's ideal for artificial intelligence stuff.

Something funny about LISP though…it has things called “predicates,” which are just its version of yes/no questions about something else. Predicates often end with the letter “p” to indicate that they are, in fact, predicates. For instance, “evenp” is the predicate that will tell you whether a variable is an even number or not. “evenp 6″ would return “true.” Anyway, that's not the funny part. The funny part is that one day recently, I was riding home from school, and as I passed one of the cars parked by the high school near our house, I saw something written on the back window in soap or that car paint or whatever it is, and it said something like:

promp mindy

I could have read it wrong…it might have been a different name, or different letters, or anything. But that's what it looked like. I thought it could possibly have been literally the most nerdy invitation to the prom in existence. And if it wasn't, then it definitely should have been.

Other than that, work has been going well, and I'm still really enjoying my job. I'm sure I'll be there for a long time yet. I'll also be going back to VA as soon as the quarter ends, on the 12th to be exact. I'll only stay for a couple weeks this time though, and then Courtney's coming back with me for until August! It's going to be awesome.

That's all for now.