in Search

cruizer

aspiring to free and open the mind of .NET developers

April 2007 - Posts

  • TDD in PHP presentation

    Yesterday I had the privilege of speaking about Test Driven Development in front of PHP developers. I had a bit of trouble at first with the projector not playing nice with my cheapo notebook. Too bad that we didn't have much time to cover necessary ground.  :( As promised, here are the links to the slides:

    The demo code is here. I'll be following this up with another post containing the continuation of the demo, using mock objects for the database and the email server.

    Feel free to post questions in the www.phpugph.com thread

  • I hate PHP gotchas

    I like PHP, but some of its gotchas are really getting on my nerves. I wonder if any of you have encountered this particular issue that's been causing my hair to fall off lately.

    One system I did for a client was generating badly-formed reports in CSV format. The mysterious thing was that it worked OK (even with their test data) on my home system using PHP 5.1.6.  I had to create my own fputcsv() function (for writing a row of data into a file in CSV format) because the version used by the client in production (together with some other web app system which I will dare not touch) is PHP 5.0.4.  Even when I forced my home machine to make use of the home-made fputcsv() function, my home machine seemed to generate OK reports.

    What I noticed with the generated CSV files (from my client's server machine) was that some characters near the end of the comma-separated row were not present. As a result when the file is loaded into Excel, multiple records (rows) are concatenated into a single row, causing it to exceed Excel's 256-column limit and throw up some error message.

    Until now I could not find the reason why it's behaving that way on PHP 5.0.4. A snippet of my home-made fputcsv function is shown below:

    $lineToWrite = '';
    foreach ($inputArr as $columnValue) {
    $columnValue = str_replace('"', '""', $columnValue);
    if ($lineToWrite != '') $lineToWrite .= ',';
    $lineToWrite .= '"'.$columnValue.'"';
    }
    fputs($fp, $lineToWrite, strlen($lineToWrite));
    fputs($fp, "\n");

    I changed it to:

    $lineStarted = false;
    foreach ($inputArr as $columnValue) {
    $columnValue = str_replace('"', '""', $columnValue);
    if ($lineStarted) {
    fputs($fp, ',');
    } else {
    $lineStarted = true;
    }
    fputs($fp, '"');
    fputs($fp, $columnValue, strlen($columnValue));
    fputs($fp, '"');
    }
    fputs($fp, "\n");

    And it worked. For some reason it didn't want to display the whole string ($lineToWrite). At first I thought that the fputs or fwrite function have some sort of maximum string length limit, but there's no mention at all of that in the PHP documentation. Since the string being written to the CSV file had some of its latter characters chopped off, the opening double quote (") delimiting a column value is not being properly matched with a closing quote. The result is that the next column (first column of the succeeding row) is mistakenly taken to be part of the last column of the previous row. To solve the problem, I no longer concatenate the entire comma-separated row into a single string, and I write out the comma (column delimiters) and the opening and closing double quotes per column value in separate fputs() calls. Ugh. I really hate PHP gotchas.

    Posted Apr 26 2007, 04:11 PM by cruizer with no comments
    Filed under:
  • Thanks, Keith!

     The picture says it all...

    Vista Ultimate! 

    Posted Apr 20 2007, 05:39 PM by cruizer with 3 comment(s)
    Filed under:
  • TDD is even more important when using dynamic languages

    I made that observation, while doing PHP TDD-style, in my previous post. In this interview with Dave Astels and Steven Baker, they also note that TDD is a necessity in dynamic languages such as Ruby. Let me quote:

    It (the compiler, in statically-typed languages) is, it's testing that you are passing the right types around, and then things are hanging together that way, you're passing the right number arguments you have the method spelled properly, it's doing all those checks. Something that Ruby is dynamically typed you don't have those checks ahead of time. That happens at runtime, you call a method and if you misspell a method name you get a "no method defined". And that's where your tests come in; they nail down all that behavior.

    It's a cool interview; those bandwidth-starved among us who don't like to watch geeks speaking can just read the text transcript. They also discuss Behavior-Driven Development or BDD. In my opinion, BDD is just TDD with a different orientation. They just call it "specifications" instead of "tests." In a way, I agree with them that too much focus on "testing" with TDD can be unhealthy because TDD is a design approach and is not a full-blown testing methodology. Nowadays when I write code I think of my test code as specifications or behavior that my actual code must fulfill.
  • Test-First Demo: Developing a User Login Facility, part 1

    OK, so now it's time to "walk the talk." I'll be demo-ing here how to develop a user login facility test first -- how you can begin a test, then code your class to pass that test. Then we'll add one feature or behavior at a time until we completely fulfill the specifications or requirements. Hopefully at the end of this article those who have not adopted test-first development yet can have a clearer idea as to how they can use the approach in their daily programming lives.

    The problem at hand here is to develop a user login facility. This is kinda common in most projects that we make and we tend to "reinvent the wheel" for every project. Anyway, let's start things simple.  Let's say our client (or our boss) has given us the following requirements for now (which means these requirements may change, maybe in Part 2, heh heh):
    1. The system will have a user login facility.  The system should count the number of consecutive failed logins for a given user account.  If the user fails to provide the proper password three consecutive times, the user account should be locked out, meaning no one can use it to log in.
    2. A successful login will reset the consecutive number of failed logins to zero.
    3. When a user account gets locked, an email should be sent to the address admin@domain.org, informing the admin about the failed login attempt by the user as well as the date and time of the attempt.  The email should also be sent whenever a user attempts to log in to an already locked account.
    Now that sounds easy, doesn't it? As I said in my previous blog entry, the hard part with test-first development is knowing what to test first. Personally, I start with the simplest requirement or specification. It's not listed in the enumeration above, but definitely an implied specification is that the system should be able to detect the entry of a wrong user password. So maybe we can start testing that.

    Ok. Let's create a class, say UserTests.cs, and let's test the entry of a wrong password. For simplicity's sake, let's not bother with user record persistence (to a database) yet. Let's just put in the user name and the password in the constructor of (the yet-unwritten) User class, then have a method VerifyLogin(string passwordToCheck):

    /* UserTests.cs */
    using
    System;
    using NUnit.Framework;

    namespace TestFirstPart1
    {
        [TestFixture]
        public class UserTests
        {
            [Test]
            public void ShouldVerifyPasswordProperly()
            {
                User user = new User("username", "correctPassword");
                Assert.IsTrue(user.VerifyLogin("correctPassword"));
                Assert.IsFalse(user.VerifyLogin("wrongPassword"));
            }
        }
    }


    Your IDE is probably complaining that the User class and its constructor and VerifyLogin() method don't exist yet. Ok, so let's give it what it wants, just enough so we can compile:

    /* User.cs */
    using
    System;

    namespace TestFirstPart1
    {
        public class User
        {
            private string name = String.Empty;
            private string correctPassword = String.Empty;

            public
    string Name
            {
                get { return name; }
            }

            public User(string username, string password)
            {
                name = username;
                correctPassword = password;
            }

            public bool VerifyLogin(string passwordToCheck)
            {
                return true;
            }
        }
    }


    Why do I have the sinking feeling that this is gonna fail? Because we just did the absolute minimum for this thing to compile. Obviously, running the test (using, say, NUnit's GUI runner tool) results in a red or failure. So let's put in the proper code for the VerifyLogin() method to pass the test:

    /* User.cs */
            public
    bool VerifyLogin(string passwordToCheck)
            {
                return (passwordToCheck.Equals(correctPassword));
            }


    Ok so now it passes. So we've got one (implied) requirement taken care of; we have three to go. Let's start with #1. This means we need to introduce a way for the User class to note how many consecutive failed logins it has been subjected to. So let's go and add properties to our User class...

    No no no Stick out tongue We were supposed to do things test first, right? Uh, ok (grudgingly) why don't we just start with a new test case:

    /* UserTests.cs */
            [Test]
            public void ShouldCountConsecutiveFailedLogins()
            {
                User user = new User("username", "correctPassword");
                Assert.AreEqual(0, user.ConsecutiveFailures);
            }


    Obviously this is going to fail because we're introducing a new property named ConsecutiveFailures. Let's just put in the bare minimum to User.cs to allow this thing to compile:

    /* User.cs */
            private
    int consecutiveFailures = 0;

            public int ConsecutiveFailures
            {
                get { return consecutiveFailures; }
            }


    Ok, the tests pass. Now let's expand our ShouldCountConsecutiveFailedLogins() test code:

    /* UserTests.cs */
            [Test]
            public void ShouldCountConsecutiveFailedLogins()
            {
                User user = new User("username", "correctPassword");
                Assert.AreEqual(0, user.ConsecutiveFailures);
                user.VerifyLogin("wrongPassword");
                Assert.AreEqual(1, user.ConsecutiveFailures);
                user.VerifyLogin("wrongPasswordAgain");
                Assert.AreEqual(2, user.ConsecutiveFailures);
            }


    Now the test fails. That's good, because it means we're on track to add functionality to User.cs to enable this feature. Basically we need to insert code into the VerifyLogin() method to count the consecutive failures everytime the password is incorrect:

    /* User.cs */
            public
    bool VerifyLogin(string passwordToCheck)
            {
                bool correct = passwordToCheck.Equals(correctPassword);
                if (!correct)
                {
                    consecutiveFailures++;
                }
                return correct;
            }


    Ok, test result is green. Let's add another test to fulfill requirement #1 completely:

    /* UserTests.cs */
            [Test]
            public void ThreeConsecutiveFailuresLockAccount()
            {
                User user = new User("username", "correctPassword");
                Assert.IsFalse(user.Locked);
                user.VerifyLogin("wrongPassword");
                user.VerifyLogin("wrongPassword");
                user.VerifyLogin("wrongPassword");
                Assert.IsTrue(user.Locked);
            }


    Don't compile this yet, since it will fail because of that new property "Locked." Obviously it's a Boolean, so we adjust User.cs accordingly:

    /* User.cs */
            private
    bool locked = false;

            public bool Locked
            {
                get { return locked; }
            }


    Running the test results in a failure again. Where do you think do we need to insert code in User.cs to make this test pass?

    /* User.cs */
            public
    bool VerifyLogin(string passwordToCheck)
            {
                bool correct = passwordToCheck.Equals(correctPassword);
                if (!correct)
                {
                    consecutiveFailures++;
                    locked = (consecutiveFailures >= 3);
                }
                return correct;
            }

     
    Ok, all three tests pass. You might even want to Assert.IsFalse(user.Locked) after each of the first two user.VerifyLogin() calls, just to make sure that the VerifyLogin() method doesn't lock the account prematurely:

    /* UserTests.cs */
            [Test]
            public void ThreeConsecutiveFailuresLockAccount()
            {
                User user = new User("username", "correctPassword");
                Assert.IsFalse(user.Locked);
                user.VerifyLogin("wrongPassword");
                Assert.IsFalse(user.Locked);
                user.VerifyLogin("wrongPassword");
                Assert.IsFalse(user.Locked);
                user.VerifyLogin("wrongPassword");
                Assert.IsTrue(user.Locked);
            }


    The updated test should still pass. So we got requirement #1 nailed down!

    Requirement #2 should be easy to do. As always, let's start with a test:

    /* UserTests.cs */
            [Test]
            public void CorrectLoginShouldResetConsecutiveFailureCount()
            {
                User user = new User("username", "correctPassword");
                Assert.AreEqual(0, user.ConsecutiveFailures);
                user.VerifyLogin("wrongPassword");
                Assert.AreEqual(1, user.ConsecutiveFailures);
                user.VerifyLogin("wrongPasswordAgain");
                Assert.AreEqual(2, user.ConsecutiveFailures);
                user.VerifyLogin("correctPassword");
                Assert.AreEqual(0, user.ConsecutiveFailures);
            }


    So you make clear here that upon logging in with a correct password, the number of consecutive failures are reset to zero.  No need to do anything yet to the User class; running the test fails though (as expected).  So what do we need to do? We need to handle a correct login and reset consecutiveFailures accordingly:

    /* User.cs */
            public
    bool VerifyLogin(string passwordToCheck)
            {
                bool correct = passwordToCheck.Equals(correctPassword);
                if (correct)
                {
                    consecutiveFailures = 0;
                }
                else
                {
                    consecutiveFailures++;
                    locked = (consecutiveFailures >= 3);
                }
                return correct;
            }


    Note that we modified the if logic here. For me it's more readable to test if (correct) than if (!correct). But that's just me anyway. All other tests work fine. We've nailed requirement #2 as well! Now to move on to the more difficult part...

    How do you test if an email is sent? That's kinda hard...especially if you don't have a mail server available for testing! What I do when confronted with such a situation is to mock the email send process. This means that I put in a different object for testing, one that does not actually send an email. What I am testing here is that the action that sends the email is actually being correctly triggered by my code (in this case, the User class). Later I can just do as part of an integration test suite the class that actually sends out email messages. And this is the code that I will be putting in my production code.

    I use that approach whenever I deal with external dependencies. In this case, that external dependency is the email server, which I may or may not have. I may have one but it's not nice to keep on sending email while I'm fleshing out my application. So in this case, I hide my action behind an interface, say IUserLockNotifier.  The IUserLockNotifier will simply define a method by which an action can be triggered.  I will use a mock class for my testing, say MockUserLockNotifier, but in my production code I will use the actual mail sending class, say UserLockEmailNotifier.

    I can choose to make use of a mocking framework such as Rhino Mocks or NMock2 to create a mock object for me automatically, or I can write my own mock implementation.  Let's choose the former, since I am more likely to make use of such tools in the "real world" anyway.  Let me start with my test:

    /* UserTests.cs */
            [Test]
            public void LockedUserTriggersAnAction()
            {
                MockRepository mocks = new MockRepository();
                IUserLockNotifier mockNotifier = mocks.CreateMock<IUserLockNotifier>();
                User user = new User("username", "correctPassword");
                user.NotifierWhenLocked = mockNotifier;
                mockNotifier.SendMessageAbout(user);
                mocks.ReplayAll();
                user.VerifyLogin("wrongPassword");
                user.VerifyLogin("wrongPassword");
                user.VerifyLogin("wrongPassword");
                mocks.VerifyAll();
            }


    By the way I also put in "using Rhino.Mocks;" near the top of the test class and add Rhino.Mocks.dll as a reference to my project.  Let's explain what we're trying to do here a bit. Basically we're creating a MockRepository. A MockRepository is a factory of mock objects.  We create an object called mockNotifier, which is a mock object implementing the IUserLockNotifier interface, which we thought of a few paragraphs ago. Note that we have not defined this interface yet...we will in a little while. We pass this mockNotifier to the NotifierWhenLocked property of the User class. Since the mock framework is in "recording mode," any calls that we make to the mockNotifier are "recorded" and noted by the MockRepository. These calls/actions will be remembered and expected later when we shift to the "play mode," which we do when we call mocks.ReplayAll().  At this point, the mocking framework will indeed expect that the SendMessageAbout() method of the notifier will be called.  This is supposed to be triggered during the third failed login, but of course we haven't modified the User class yet to make use of this notifier.

    Definitely, compiling this code will fail since we have not defined IUserLockNotifier interface and the NotifierWhenLocked property of the User class yet. Let's put the bare minimum to get things going:

    /* IUserLockNotifier.cs */
    namespace
    TestFirstPart1
    {
        public interface IUserLockNotifier
        {
            void SendMessageAbout(User user);
        }
    }


    /* User.cs */
            private
    IUserLockNotifier notifierWhenLocked = null;

            public IUserLockNotifier NotifierWhenLocked
            {
                set { notifierWhenLocked = value; }
            }


    Now that should compile. Running the test, we see a red, and we see the actual error message from the test runner:

    TestFirstPart1.UserTests.LockedUserTriggersAnAction : Rhino.Mocks.Exceptions.ExpectationViolationException : IUserLockNotifier.SendMessageAbout(TestFirstPart1.User); Expected #1, Actual #0.
       at Rhino.Mocks.Impl.ReplayMockState.Verify()
       at Rhino.Mocks.MockRepository.Verify(Object obj)
       at Rhino.Mocks.MockRepository.VerifyAll()
       at TestFirstPart1.UserTests.LockedUserTriggersAnAction() in C:\WhateverDirectory\Projects\TestFirstPart1\TestFirstPart1\UserTests.cs:line 67

    Basically this means that an expectation failed. Rhino Mocks expected that the SendMessageAbout() method of the mockNotifier will be called once (Expected #1) but wasn't at all (Actual #0). Of course, this is because we have not modified User.cs to actually make use of the notifierWhenLocked reference to an IUserLockNotifier.  This should be triggered when a user account gets locked. So we need to modify User.VerifyLogin() to become:

    /* User.cs */
            public
    bool VerifyLogin(string passwordToCheck)
            {
                bool correct = passwordToCheck.Equals(correctPassword);
                if (correct)
                {
                    consecutiveFailures = 0;
                }
                else
                {
                    consecutiveFailures++;
                    locked = (consecutiveFailures >= 3);
                    if (locked && notifierWhenLocked != null)
                    {
                        notifierWhenLocked.SendMessageAbout(this);
                    }
                }
                return correct;
            }


    The tests now pass. So are we done? Hmm...take a look at the requirements list above and you'll notice that we should be able to send a message to the notifier (via email) whenever an attempt is made to log in to a locked account. Obviously if an account is locked, we shouldn't allow that user to log in, even if he/she is now able to put in the correct password. After all, the cracker might be doing a brute-force password guess attack and he/she might have finally guessed the correct password.  So another test is in order:

    /* UserTests.cs */
            [Test]
            public void AnyLoginWhenLockedTriggersNotifierAction()
            {
                MockRepository mocks = new MockRepository();
                IUserLockNotifier mockNotifier = mocks.CreateMock<IUserLockNotifier>();
                User user = new User("username", "correctPassword");
                user.VerifyLogin("lockMeFirst");
                user.VerifyLogin("lockMeFirst");
                user.VerifyLogin("lockMeFirst");
                Assert.IsTrue(user.Locked);
                user.NotifierWhenLocked = mockNotifier;
                mockNotifier.SendMessageAbout(user);
                mocks.ReplayAll();
                bool result = user.VerifyLogin("correctPassword");
                mocks.VerifyAll();
                Assert.IsFalse(result);
            }


    Ok, so what do we have here? Let me explain first. First, we are again creating a mock IUserLockNotifier called mockNotifier. Note that we don't assign it yet to the NotifierWhenLocked property of the User we created.  This is because we want to make sure that the account is locked first...so we "login" three times with a wrong password to make sure it's locked -- in fact we do an Assert.IsTrue() to make sure the user account is really already locked. So this is when we inject our mockNotifier into the NotifierWhenLocked property.  We set the expectation that the notifier object's SendMessageAbout() method should be called.  Then we move out of the mock recording mode and into mock replay mode.  The VerifyLogin() call with the correct password should still trigger an email send and result in a failed login -- because the user account has already been locked.

    Running this test obviously fails. We need to modify VerifyLogin() again to trigger the notification action when a login attempt to a locked account is made, regardless of the correctness of the password:

    /* User.cs */
            public
    bool VerifyLogin(string passwordToCheck)
            {
                if (locked)
                {
                    if (notifierWhenLocked != null) notifierWhenLocked.SendMessageAbout(this);
                    return false;
                }
                bool correct = passwordToCheck.Equals(correctPassword);
                if (correct)
                {
                    consecutiveFailures = 0;
                }
                else
                {
                    consecutiveFailures++;
                    locked = (consecutiveFailures >= 3);
                    if (locked && notifierWhenLocked != null)
                    {
                        notifierWhenLocked.SendMessageAbout(this);
                    }
                }
                return correct;
            }


    So now the tests pass. Smile Looks like our VerifyLogin() method has grown a bit long. If you don't like it, you can refactor it. The good thing about it is that we now have a test harness that we can run and re-run while refactoring. As long as the tests pass, you know that you're still meeting the requirements or specifications set by your customer.  I leave the exercise of refactoring VerifyLogin() to you.

    Ok so we have a mock notifier, but that still does not meet the requirement that it should actually be able to send email. So let's show how you can make a simple class that performs the email action:

    /* UserLockEmailNotifier.cs */
    using
    System;
    using System.Net.Mail;
    using System.Text;

    namespace TestFirstPart1
    {
        public class UserLockEmailNotifier : IUserLockNotifier
        {
            private string emailTo = String.Empty;
            private string mailServer = String.Empty;

            public UserLockEmailNotifier(string emailToNotify, string mailServerToUse)
            {
                emailTo = emailToNotify;
                mailServer = mailServerToUse;
            }
            public void SendMessageAbout(User user)
            {
                MailMessage message = new MailMessage("watcher@domain.org", emailTo);
                message.Subject = String.Format("Attempt to login with locked user {0}", user.Name);
                StringBuilder body = new StringBuilder();
                body.Append("Hi, \n");
                body.AppendFormat("An attempt was made to login using the locked user account {0} on {1}.",
                                  user.Name, DateTime.Now.ToLongDateString());
                body.AppendLine();
                body.AppendLine("Please take necessary precautionary measures. Thank you.");
                message.Body = body.ToString();
                SmtpClient client = new SmtpClient(mailServer);
                client.Send(message);
            }
        }
    }


    To be able to do an automated integration test for this, you will need to have programmatic access to a mail account inbox (via POP3 or IMAP, for example) and check to see it receives an actual message. It's kinda complicated since you'll have to take into account possible delay or time gap between sending the message to the SMTP server and actually receiving it in the destination account inbox.  So for now let's leave this thing without an integration test Stick out tongue In your actual code, you can do something like this snippet from the integration test code:

    /* UserIntegrationTest.cs */
    using
    System;
    using NUnit.Framework;

    namespace TestFirstPart1
    {
        [TestFixture]
        public class EmailIntegrationTests
        {
            [Test]
            public void ShouldSendEmailWithoutError()
            {
                User user = new User("username", "correctPassword");
                UserLockEmailNotifier emailNotifier = new UserLockEmailNotifier("admin@domain.org",
                                                                                "smtp.domain.org");
                user.NotifierWhenLocked = emailNotifier;
                user.VerifyLogin("failedLogin");
                user.VerifyLogin("failedLogin");
                user.VerifyLogin("failedLogin");
            }
        }
    }


    The test will pass if an exception does not occur (which will, if there's something wrong with the SMTP server at smtp.domain.org or if it does not exist).

    I hope you were able to learn something from this short demo. I'll see if I can create a screencast/video version of this demo. In part 2 we'll add to the list of requirements to do things like database interaction, so stay tuned!