dl

David Lundgren

Web Developer & Systems Administrator

The allure of static proxies

Several weeks ago I started playing with Laravel. Primarily because several colleagues are using it, and have suggested that I take a look at it. During my time reviewing how to build a view template I came across references to Html, Form, View and other static calls. Initially I was not impressed due to the use of so many static calls. I have come to an understanding about how static calls in certain circumstances can actually enhance code readability.

Continue reading

Could not generate DH keypair on “Add composer dependency” in PhpStorm

If you are like me and you run an older version of OS X, Mavericks in my case, it means that you may run into the problem with PhpStorm that you can’t use composer. It seems like you are stuck in this rut. Well, I’m going to put this here mainly for my own memory, and in the hopes that it brings someone else luck in getting PhpStorm working correctly.

https://youtrack.jetbrains.com/issue/WI-27354#comment=27-999854

pure-authd external script failing

I have a PHP script handling authentication for pure-authd, and the problem is that it fails. After a reboot it is still failing. If I log in, restart it manually and monitor the log files, it works. As soon as I log out it fails!

I’ve come to the conclusion that pure-authd doesn’t properly daemonize, and detach from the controlling terminal. PHP was actually using fsockopen to another server and it stopped being able to use DNS because I logged out of the shell.

I ended up switching to the IP of the server. While this is not the ideal solution, it works because internally the IP is a reserved DHCP address for the server, and we shouldn’t be updating the server any time soon.

SPL FileObject & LimitIterator

Over that last couple of weeks I’ve come to use the SPL far more than I have in the past. The SplFileObject for reading CSV files is far more convenient than the fgetcsv() function, and associated code needed for a CSV file. Using the LimitIterator allowed me to easily bypass the first row of the CSV, as they were headers and I knew the format of those headers.

$csv = new \SplFileObject("some-file.csv");
$csv->setFlags(\SplFileObject::READ_CSV | \SplFileObject::READ_AHEAD |
               \SplFileObject::SKIP_EMPTY | \SplFileObject::DROP_NEW_LINE);
foreach (new \LimitIterator($csv, 1) as $row) {
    /** process the rows **/
}

That was fast for iterating the CSV, the previous code I had for doing this was more verbose about processing for the header row, making it a little more error prone. Take a look at the setCsvControl() method on SplFileObject for more fine grained control over the delimiter, enclosure and escape parsing.

The SPL LimitIterator has also come in handy in another project. This time though I was using a SQL union in a query to merge two separate datasets and I only wanted to list the first 12 out of a potentially larger number of objects.

$sql = '(SELECT id, somestuff FROM somedb.sometable)
        UNION
        (SELECT id, otherstuff FROM otherdb.othertable) ORDER BY `date` DESC';
$data = [];

// $db is a ZF1 adapter object
foreach ($db->query($sql) as $row) {
    $data[$row['id'][] = $row;
}

foreach ($data as $id => $d1) {
    foreach (new \LimitIterator(new \ArrayIterator($data), 0, 12) as $id => $row) {
        /** process the rows **/
    }
}

I didn’t use the following code in the end, but it did work. I suspected, but didn’t confirm, that MySQL’s GROUP BY MAY introduce unknown bugs to the data imported, so I used the above method instead. Here is an example of how the SPL MultipleIterator could be used:

$sql = 'SELECT id, GROUP_CONCAT(`type`) `type`,
               GROUP_CONCAT(`date`) `date`,
               GROUP_CONCAT(`path`) `path`,
               GROUP_CONCAT(`synced`) `synced`
        FROM somedb.sometable
        GROUP BY id, `date` DESC';
foreach ($db->query($sql) as $row) {
    $id = $row['id'];
    // build our array iterators
    $typeArray = new \ArrayIterator(explode(',', $row['type']));
    $dateArray = new \ArrayIterator(explode(',', $row['date']));
    $pathArray = new \ArrayIterator(explode(',', $row['path']));
    $syncArray = new \ArrayIterator(explode(',', $row['synced']));

    // build the main iterator
    $iterator = new \MultipleIterator(\MultipleIterator::MIT_KEYS_ASSOC);

    // make the array iterators combinable into a single array entry
    $iterator->attachIterator($typeArray, 'type');
    $iterator->attachIterator($dateArray, 'date');
    $iterator->attachIterator($pathArray, 'path');
    $iterator->attachIterator($syncArray, 'synced');

    // The array keys of $data will be as set in the attachIterator()
    // method of the MultipleIterator
    foreach(new \LimitIterator($iterator, 0, 12) as $data) {
        /** process the data **/
        if ($data['synced'] == (int)1) {
            continue;
        }
        $data['type']; 
        $data['date'];
        $data['path'];
    }
}

Take a look at the Standard PHP Library for ideas on using the other iterators, data structures, and objects that bring potential for more readable code into your life. They’ve simplified my life a bit because now I don’t have to keep track of variables just to handle counts and other things for state.

When does Dependency Injection become an anti-pattern?

During my tenure as a seasoned, and tenderized, PHP developer I have used many design patterns: adapters, factories, data mappers, facades, etc. The most recent one that I have been working with is Dependency Injection. Inversion of Control is not a new idea, at least not in the programming world, but in the PHP world it seems to have taken us by storm in recent years. Every framework will often have a Dependency Injector built in, or offer it as a component for use. It is the hip thing to do in PHP, but I believe we need to take a step back and evaluate it for what we are really trying to achieve. That is reducing the tight coupling that our objects may have. I view it as removing the new-able’s from our objects code, and handing the object creation over to something else to deal with. Read further on how I believe DI can become an anti-pattern.

Continue reading

BlockList.de IP lists with CIDR notation

Several weeks ago one of the servers I manage ended up with a lot of comment spam. After several hours of searching through the logs and correlating the IP’s, I found out that a majority of the hosts were listed in the BlockList.de’s bots.txt file. I didn’t want to import ALL the IP’s on the list as I noticed that quite a few could be pushed into fewer lines using CIDR notation.

I set out to compress this and ended up with this Gist that can do exactly that.

<?php
require_once "/path/to/BlocklistDe.php";

$botList = new BlocklistDe('bots.txt');
$botList->writeToFile('converted-bots.txt');

This particular server was using Ubuntu, so I pushed the 13k addresses into the iptables configuration. Spam has been cut quite dramatically.

ZfRest — RESTful-ness for Zend Framework 1

For both a client and my work we are working on creating a RESTful API. The problem is that all of the current solutions for Zend Framework are either one off (Resauce, ZfApiVersioning) or otherwise do not play well with a pre-built Zend Framework 1 application. The solution I was looking needed the following

  • handle API versions in the route (I know this isn’t in the spirit of RESTful)
  • handle OAuth
  • Plays well with ZF1 Module architecture (using a Module Bootstrap)

REST modules

In the end most of the solutions out there handle one thing or another but do not address the playing well with ZF1. Resauce deletes the default routes and sets it’s own root url (‘/’), this is not acceptable when you are trying to integrate it with an existing application. ZfApiVersioning suffered the same thing. The only available solution that I found was to write my own REST library for ZF1, so I am introducing ZfRest. It handles the requirements above, and more. The OAuth part is actually loaded as a plugin, requires PECL Oauth, and provides a basic way to handle using Oauth.

OAuth

I also needed an OAuth provider or server. Out of the few solutions I found for ZF1 (link: Zend_Oauth_Provider [incorrect naming], link: Oauth_Server [too complex]), I opted to just use the PECL OAuth module. I quickly ran into a problem where the Exceptions from an internal function were not being passed to my script when you pass in a path such as “/api/users”. Once I figured this out, everything was just working.

$oauthProvider = new OauthProvider();
$oauthProvider->consumerHandler('lookupConsumer');
$oauthProvider->timestampNonceHandler('lookupTimestampNonce');
try {
    $oauthProvider->checkOAuthRequest($uri, $method);
}
catch (Exception $e) {
    // Set HTTP response code to 403
    echo json_encode(array('statusCode' => 403, 'message' => 'Access denied to this resource'));
}

I’ve had several updates to problems I’ve found while using it, but overall I’ve found it works rather well.

PHPUnit and certainty of test cases with random data

Recently I was testing a function that generated slugs for an application. In order to make the slugs unique we would append the microtime to the slug if needed. After updating my data provider to account for using the microtime version of the slug I was receiving intermittent success for the test cases. Using microtime for generated data introduces a margin of error into the time between when the code is executed and when you compare the results.

I tried using microtime before and after the function call that generated the slugs, but I would still end up around ~0.0010 seconds behind the microtime that the slug generator was using. I could not figure out how to make these tests pass 100% of the time, within reason.

If you can’t control the generator for tests that involve random data you have 2 options:

  1. refactor to remove the randomness
  2. live with a degree of certainty

For the purposes of generating a slug with appended microtime, I determined that the degree of certainty was that the slug’s microtime at the seconds level will either be equal to or at most one second before the microtime of the test cases microtime call. If it is greater than 1 second difference then there is definitely a problem. The 1 second threshold could most likely be reduced (for example to 0.0010 seconds) if desired, but I needed to get the test written in a timely manner and a 1 second degree of certainty was acceptable at the moment.

Redux: PHP5 cronjob in Debian packages

After running into “session could not be started because it was already started with session_start() or session.auto_start” on a project, I realized that removing the cronjob is not the only thing that needs to happen to let PHP manage it’s own sessions.

  • chmod www-data:www-data /var/lib/php5
  • update /etc/php5/apache2/php5.ini and set
    session.gc_probability = 1

I’m assuming that this more paranoid than usual security measure was a way to help inexperienced admins and developers to help prevent session hijacking should the web server be breached. However, if root is gained, it doesn’t matter anyway. I’m not going to say that I am an expert in security for servers, but I can tell you that Debian, and therefore Ubuntu, are the only ones doing this type of paranoid security practice. Coming from the FreeBSD world, you are responsible for the security of your machine, not the developers or port maintainers.

</rant>

Zend Framework 1.11 + PHPUnit 3.6 unit testing

At work we are up to date with PHPUnit  at 3.6, yet officially ZF1 will only support PHPUnit 3.4. Looking through the code there is some support for PHPUnit 3.5, but I actually make use of the additional assertions that PHPUnit 3.6 provides. Initially I had intended on just extending the Zend_Test PHPUnit classes but I quickly ran into the problem where the ZF classes didn’t implement methods in the interfaces when they were brought into the PHP interpreter. I then thought about copying over the Zend_Test classes and renaming them, this would have created a maintenance issue since we don’t want to maintain our own fork of the Zend_Test classes.

My final solution was simply to create my own DatabaseTestCase. This is what I came up with:
Continue reading

Next Page »