Posts Tagged ‘library’

PDOQuery: jQuery Style Database Access and Manipulation

PDOQuery Logo
No Gravatar

I’ve been working on a new database library recently. It’s written in PHP, since that’s what I need, but nothing says it couldn’t be ported to other languages (possibly Java, Python, or Ruby). I’ve had a de-facto library that I’ve used for my projects for nearly six years, but it just isn’t cutting it anymore.

My old one is a relatively simple generic class with standard CRUD functions for batch and single primary-key-based operations. I wrote it myself when I first learned about object-oriented programming, and I’ve made a few tweaks here and there through the years. It’s just too heavy though, and not very intuitive. Anyone unfamiliar with my code would really have to study the library for a long time to figure out how to use certain parts—I am sure of this because even I have to really study it sometimes to refresh my memory. That’s a bad sign when the library in question is a simple database access layer.

Enter PDOQuery. PDOQuery is my new library. It’s modeled to a limited extent after jQuery syntax, though obviously not all the same kinds of functionality apply, and the language difference (PHP vs. Javascript) means that PDOQuery uses -> for chaining instead of a period. The query language is built in a similar fashion, only PDOQuery selects and manipulates a database instead of DOM elements.

It’s also meant to give you as much power as possible with as little pain as possible. Query modifications are built to return a PDOQuery object, just like jQuery, so you can link modifications together or apply a new selector to an existing object. The library takes full advantage of PHP to provide powerful object integration, making it extremely simple (and mostly automatic) to create a PHP class that directly represents a database table row. It also supports table references, so with a single line of PHP, you can select a user and all related tasks, for example.

Consider the following implementation code:

include 'PDOQuery.class.php';
PQ() -> connect("mysqli:dbname=testdatabase", "testuser", "testpass");

The PQ() function is a shortcut to the main PDOQuery class. It takes an optional selector argument and returns an instance of the class. Note that this single include and call to the connect() method are the only bits of code necessary to start using the library. Now for some usage:

$s = PQ("users.join:permissions[UserID]");
$users = $s -> select();

The first line above creates a PDOQuery object that parses the selector into appropriate MySQL query, but it doesn’t actually run the query. The second line is what runs it and stores the result. For the MySQL database type (the only one coded at this point), the default result returned is a native MySQL result. You can optionally request a numerically indexed array of rows, an associative array of rows, a numerically and associatively indexed array of rows, or an array of objects. The native result is the default because it has the smallest memory requirement.

You could also run the following code:

$s -> delete();

…though I wouldn’t recommend it. Because the selector in the $s object is parsed and stored in individual pieces (table name, criteria, joins, etc.), it doesn’t assume you’re going to just be selecting. The line above would use all of the selector parts applicable to a DELETE query (just the table name, in this case), ignoring all the rest (the JOIN option), and run the following MySQL query:

DELETE FROM `users`

You could also do the following:

$s2 = $s -> where("UserID=3");
$s2 -> delete();

This would delete only the user whose UserID is equal to 3. Much safer.

Currently, the PDOQuery library supports the following database-related methods:

  • query()
  • select()
  • selectRow()
  • selectColumn()
  • selectValue()
  • insert()
  • update()
  • delete()

The query() method is used by all the others, and can be used by you to run a raw query if you cannot accomplish it using other methods. It returns a native database result. The selectRow() method returns a single row, selectColumn() returns an array of values from a single column and multiple rows, and selectValue() returns the value of one column’s content from within a single row.

PDOQuery also has these chainable selection methods:

  • join()
  • naturaljoin()
  • leftjoin()
  • rightjoin()
  • outerjoin()
  • leftouterjoin()
  • rightouterjoin()
  • where()
  • groupby()
  • orderby()
  • having()
  • limit()

The outerjoin() method is a the exact same as leftouterjoin(). All of these return a PDOQuery object for further chaining or manipulation.

However, while this stuff is all important, it isn’t the best feature of the library. PDOQuery includes a discover() method which creates a map of fields, data types, and indexes of tables in the database. This information can then be used to generate code for you to inject into your object classes to enable many convenient shortcuts. You can add just a bit more information yourself to establish relationships between tables for even greater conveniences.

Consider a database with a users table and a projects table. A user may have zero or more projects. The following code is mostly generated by PDOQuery, except for the class names and the references index keys:

class User extends PDOQueryObject {

    public static $PDOQueryTable = "users";
    public static $PDOQueryStructure = array(
        "timestamp"=>"Fri, 25 Jun 2010 10:27:02 -0600",
        "fields"=>array(
            "UserId"=>array("type"=>"int(10) unsigned", "simpletype"=>"int", "null"=>false, "key"=>"PRI", "default"=>null, "ai"=>true, "indexes"=>array("PRIMARY"), "extra"=>"auto_increment"),
            "EmailAddress"=>array("type"=>"varchar(128)", "simpletype"=>"text", "null"=>false, "key"=>"", "default"=>null, "ai"=>false, "indexes"=>array(), "extra"=>""),
            "Password"=>array("type"=>"varchar(32)", "simpletype"=>"text", "null"=>false, "key"=>"", "default"=>null, "ai"=>false, "indexes"=>array(), "extra"=>""),
            "FirstName"=>array("type"=>"varchar(32)", "simpletype"=>"text", "null"=>false, "key"=>"", "default"=>null, "ai"=>false, "indexes"=>array(), "extra"=>""),
            "LastName"=>array("type"=>"varchar(32)", "simpletype"=>"text", "null"=>false, "key"=>"", "default"=>null, "ai"=>false, "indexes"=>array(), "extra"=>""),
            "LastLogin"=>array("type"=>"datetime", "simpletype"=>"datetime", "null"=>false, "key"=>"", "default"=>null, "ai"=>false, "indexes"=>array(), "extra"=>""),
            "Deleted"=>array("type"=>"tinyint(4)", "simpletype"=>"int", "null"=>false, "key"=>"", "default"=>null, "ai"=>false, "indexes"=>array(), "extra"=>"")
            ),
        "indexes"=>array(
            "PRIMARY"=>array("unique"=>"", "collation"=>"A", "cardinality"=>"1", "fields"=>array("UserId")),
            ),
        "primarykey"=>"UserId",
        /* USER-SUPPLIED SETTINGS */
        "references"=>array(
            "projects"=>array("key"=>"UserId", "autojoin"=>false, "relationship"=>"multiple", "class"=>"Project", "local"=>"Projects"),
            )
        );
}

class Project extends PDOQueryObject {

    public static $PDOQueryTable = "projects";
    public static $PDOQueryStructure = array(
        "timestamp"=>"Fri, 25 Jun 2010 10:27:02 -0600",
        "fields"=>array(
            "ProjectId"=>array("type"=>"int(10) unsigned", "simpletype"=>"int", "null"=>false, "key"=>"PRI", "default"=>null, "ai"=>true, "indexes"=>array("PRIMARY"), "extra"=>"auto_increment"),
            "UserId"=>array("type"=>"int(10) unsigned", "simpletype"=>"int", "null"=>false, "key"=>"", "default"=>null, "ai"=>false, "indexes"=>array(), "extra"=>""),
            "Name"=>array("type"=>"varchar(64)", "simpletype"=>"text", "null"=>false, "key"=>"", "default"=>null, "ai"=>false, "indexes"=>array(), "extra"=>""),
            "Deleted"=>array("type"=>"tinyint(4)", "simpletype"=>"int", "null"=>false, "key"=>"", "default"=>null, "ai"=>false, "indexes"=>array(), "extra"=>"")
            ),
        "indexes"=>array(
            "PRIMARY"=>array("unique"=>true, "collation"=>"A", "cardinality"=>"0", "fields"=>array("ProjectId"))
            ),
        "primarykey"=>"ProjectId",
        /* USER-SUPPLIED SETTINGS */
        "references"=>array(
            "users"=>array("key"=>"UserId", "autojoin"=>true, "relationship"=>"single", "class"=>"User", "local"=>"User")
            )
        );
}

Notice the extended PDOQueryObject class, which provides most of the convenience functions. Also notice the references key down at the bottom of each class definition. For each of these:

  • key is the correlating field name
  • autojoin determines whether to select this related information automatically
  • relationship is either “single” or “multiple” depending on whether it is 1-to-1 or 1-to-many
  • class is the name of the class that represents the joined table rows
  • local is the name of the local variable to assign the joined data

This allows us to do something like this:

$user = new User(1);

Now, $user is an object that contains a copy of the data row from the users table that has UserID equal to 1. The fields are stored in respective member variables: UserID, EmailAddress, Password, etc. These can be modified directly, since they are public. If you want to make changes, simple assign new values to the fields, then run the following code:

$user -> update();

Voila! Because that update() method is running on an instance of a single User class, it knows to treat it as a database row, and it already has all of the table structure and primary key information handy. It even performs automatic data validation and formatting (most conveniently for DATE, TIME, and DATETIME fields). You could even assign a new UserID value (or set it to NULL) to your $user object and then run:

$newID = $user -> insert();

…and you’ve got a new row inserted, just like that. But wait, there’s more! Because of the references array value, we can also do this:

$user -> loadReference("projects");

…and now you have a new Projects member variable which contains an array of objects pulled from the database and instantiated using the PDOQueryObject child Project class. If the autojoin value had been set to true, this would have been done for us automatically. Now that is convenient.

I’m still working on a few more convenience functions (data table and entry form displays, and lambda functions for automating per-row operations), and I still have to finish the documentation. It’s going to be really, really useful at least for me, and hopefully for many others. I’ll release the code as soon as I’m satisfied with it.

Rapid Android Development with AML

No Gravatar

7/13/2010 update:

Thanks to the great response (and many hundreds of visits to this page over the last two days), I’ve gone ahead and created a real project website for AML at http://www.amlcode.com, as well as the @amlcode Twitter account. If you’re interested in more updates, check those out. Thanks for all the comments and ideas! Keep them coming!

This post still has some good info and some great comments, so don’t neglect it completely. It might get lonely.

I gave myself a crash course in Android development over the last few days. I’d officially been part of an Android team as part of a project management class in my last quarter at Cal Poly Pomona, but some of the other members on the team ended up doing most of the actual development, so my instructive coding experience was pretty minimal. I knew enough to get the Android SDK installed in Eclipse (which is painfully slow, by the way!), but that was all. Everything else was new to me.

Now, I am very familiar with C, C++, and Java, even though most of my coding for my job is done in PHP, so picking up the Android vocabulary within Java was much easier for me than it would have been if I had no programming experience at all.

I don’t have a specifically defined project to work on yet, but I do have some general requirements for the Android app(s) I’m sure I will be developing in the near future. Specifically, I know I will need these features:

  1. Native implementation (Android UI, not HTML5)
  2. Dynamic externally generated content and navigation

Both of those two features together are, I believe, not a common combination. It’s simple to have dynamic externally generated content and navigation, if you use HTML and a browser. It’s also simple to have a native UI support externally generated content, if that content always fits into a pre-defined structure which you build into your app. But what if you want an app that supports a system with very little known ahead of time—just an email address and password, for instance—and the rest of it need is determined based on parameters, permissions, and a web service API?

Then, I thought, why not write a native Android library that will accommodate that very thing?

And that’s what I did.

I even had the presumptuous audacity to give it an official-sounding name, though it is by no means official (or truly useful) just yet. There is no W3C specification for it. And for all I care, it may never become official, but even if I’m the only one who uses it, this library it will certainly make things easier for me. And on top of that, it’s been a great learning experience over the last four days.

Ladies and gentlemen, I introduce to you AML, the App Markup Language, a simple XML-based language that allows you to easily build a clean, functional application for your mobile device.

It resembles HTML in some ways, but it is not a subset of HTML. It is designed to be intuitive and easy, and while it does not implement every possible feature achievable with native Android code, the goal is for it to cover the vast majority of what is necessary for information-based apps, and leave the rest up to you to implement as you desire. It probably won’t help out much with games, but it’s perfect for mobile implementations of websites. It can even be used as part of a local-only application to quickly build the UI instead of using Java code or Android’s layout XML structure.

Additionally, I would love at some point to extend AML in two directions: one, to have it support other devices (namely the iPhone and iPad), and two, to support different languages for the markup (namely JSON, perhaps YAML). While I don’t need these right now and may not actually need them for some time, there is nothing really Android-specific or XML-specific about the language. I have never written code for iOS, so I don’t know if the behavior is inherently different enough to cause a problem, but I bet it isn’t. You should be able to use the same AML markup to generate nearly identical experiences on Android and on iOS.

So, why is AML so cool? Why would anyone want to use it? Check this out.

The images on the right shows a set of different input objects in an Android app. It’s just an example, obviously—an app in the Market that looked like this would be odd indeed. However, it does illustrate a very basic layout with a few different objects.

Now, anyone who has done any Android development know that there isn’t anything truly difficult about building something like this. If you write your own “main.xml” layout file and use it for your Activity object’s content view though, it can be a little tedious. You can use a tool like DroidDraw to speed things up (which is an excellent idea if you need a static layout), but with both of these approaches, your design will be hard-coded into your app. You can only make changes on the fly through clever uses of the LayoutInflater service.

Your other option is to build the layout using only code. This allows for more dynamic design, for sure, but doing this manually is still tedious. AML bridges the gap between an external data source and the native application, so that the external source can very simply instruct the app what to build and how it should behave.

This is the Android layout XML necessary to create the layout shown above in the screenshot on the right:

<LinearLayout
    android:orientation="vertical"
    android:layout_width="fill_parent"
    android:layout_height="fill_parent">
    <Button
        android:layout_width="wrap_content"
        android:layout_height="wrap_content"
        android:text="Test Button" />
    <CheckBox
        android:layout_width="wrap_content"
        android:layout_height="wrap_content"
        android:text="Test Checkbox" />
    <ImageButton
        android:layout_width="wrap_content"
        android:layout_height="wrap_content"
        android:src="@drawable/icon" />
    <ToggleButton
        android:layout_width="wrap_content"
        android:layout_height="wrap_content"
        android:checked="true" />
    <EditText
        android:layout_width="wrap_content"
        android:layout_height="wrap_content"
        android:text="Test Text"
        android:singleLine="true" />
    <EditText
        android:layout_width="wrap_content"
        android:layout_height="wrap_content"
        android:text="Test Password"
        android:password="true"
        android:singleLine="true" />
    <RadioGroup
        android:layout_width="fill_parent"
        android:layout_height="wrap_content"
        android:orientation="vertical">
        <RadioButton
            android:layout_width="wrap_content"
            android:layout_height="wrap_content"
            android:checked="true"
            android:text="Radio Button 1" />
        <RadioButton
            android:layout_width="wrap_content"
            android:layout_height="wrap_content"
            android:text="Radio Button 2" />
        <RadioButton
            android:layout_width="wrap_content"
            android:layout_height="wrap_content"
            android:text="Radio Button 3" />
    </RadioGroup>
</LinearLayout>

Keep in mind that the above code is basically fixed, and cannot really be customized at runtime on a large scale. Now, for comparison, here is the AML necessary to create the layout above:

<aml>
  <input type="button">Test Button</input>
  <input type="checkbox">Test CheckBox</input>
  <input type="togglebutton" checked="yes" />
  <input type="imagebutton" image="drawable/icon" />
  <input type="text">Test Text</input>
  <input type="password">Test Password</input>
  <radiogroup>
    <input type="radiobutton" checked="yes">Radio Button 1</input>
    <input type="radiobutton">Radio Button 2</input>
    <input type="radiobutton">Radio Button 3</input>
  </radiogroup>
</aml>

It’s that simple. Additionally, for my tests, this layout info was actually being pulled from a web service, just so I could prove to myself that I could do it. The web service dynamically built that layout.

Here is a sample Android ListView constructed with the following AML:

<aml>
  <list>
    <item>Sample Test List Item A</item>
    <item>Sample Test List Item B</item>
    <item>Sample Test List Item C</item>
    <item>Sample Test List Item D</item>
    <item>Sample Test List Item E</item>
    <item>Sample Test List Item F</item>
    <item>Sample Test List Item G</item>
    <item>Sample Test List Item H</item>
    <item>Sample Test List Item I</item>
    <item>Sample Test List Item J</item>
    <item>Sample Test List Item K</item>
    <item>Sample Test List Item L</item>
    <item>Sample Test List Item M</item>
    <item>Sample Test List Item N</item>
    <item>Sample Test List Item O</item>
    <item>Sample Test List Item P</item>
    <item>Sample Test List Item Q</item>
    <item>Sample Test List Item R</item>
    <item>Sample Test List Item S</item>
    <item>Sample Test List Item T</item>
    <item>Sample Test List Item U</item>
    <item>Sample Test List Item V</item>
    <item>Sample Test List Item W</item>
    <item>Sample Test List Item X</item>
    <item>Sample Test List Item Y</item>
    <item>Sample Test List Item Z</item>
  </list>
</aml>
And another sample TableLayout that uses alignment and column spanning, generated by the following AML:

<aml>
  <table expand="2">
    <tr>
      <td colspan="2">This is a block of text contained within a single table cell. This cell has a colspan of 2 so that it looks good on top of the label/field pairs below.</td>
    </tr>
    <tr>
      <td align="right">Email:</td>
      <td><input type="text" width="fill" /></td>
    </tr>
    <tr>
      <td align="right">Password:</td>
      <td><input type="password" width="fill" /></td>
    </tr>
    <tr>
      <td colspan="2">Here's another 2-column cell. Is this awesome or what?</td>
    </tr>
    <tr>
      <td colspan="2">Notice the slight visual break from the text above. This text is in yet another row. The field labels above on the left are aligned to the right, and the table is instructed to make the 2nd column expandable (like an "auto-width" kind of thing).</td>
    </tr>
    <tr>
      <td colspan="2">Additionally, the fields have their widths set to "auto" so that even though they are empty, they fill their parent cells.</td>
    </tr>
  </table>
</aml>

Seriously, can it get any easier than that? You can store this kind of layout definition in your app code and reference it directly from there, or you can use a web service to build this stuff on the fly and send it to your app (though you might want to implement some sort of caching mechanism in that case, for efficiency). The AML library is in its own package, and doesn’t require any imports from your package to work. You do need to put the template layout XML files into your /res/layout folder, but that’s all. AML doesn’t require you or your web service to know the details of how Android builds its views. It doesn’t depend on some extra plugin installed on your device, and it isn’t passing your app code through another service. You probably still have to write some Java code to accomplish your goal, but this makes the whole thing easier.

So far, what I have written only supports layouts, but no actions. Many objects support attributes like align, valign, fontsize, padding, color, and bgcolor, to name a few. The rest of the code just isn’t there yet. I have been building the tap and hold attributes into buttons and other clickable objects, but I’m not done. I haven’t done a date picker, time picker, or spinner (Android’s version of an HTML <select> element). But it’s coming together very quickly—a lot faster than I thought it might. It’s been an awesome learning experience so far though.

I’m writing this for me, honestly, but I’m really interested in any kind of outside interest. Any questions, comments, or recommendations? Has anyone else done this? Would you be excited to get your hands on this code? If you used it, what would you need it to be able to do?

Let me know in the comments!