My first stabs at creating captions for memes are given below.  It was fun, I must say. I caught on to the lolcats trend a year too late.  I hope it’s still fashionable yet to post these here.  All the pictures were found on the icanhascheezeburger.com website.

Now I realize how much I didn’t know about the labs in this class.

 

Before, there was so much I didn’t know, I didn’t even know I didn’t know many things.

 

Especially after putting all this time into these couple of labs, I wish that I actually get to implement something like this again in the future, for some greater benefit. Amazon has some great technologies, and it was fun to play around with these things. After successfully using SQS and EC2, I have come to a greater appreciation for the power behind these technologies. I’m glad that I have had experience building the components of a multi-tiered system. It’s just something that I haven’t done before. It’s cool to play around with configurations (install other people’s stuff and enslave that stuff to make it do what I want!)

 

Anyway, enough on that; moving on.

 

I’m glad I had a couple of the following tools to help me out. I suppose the lesson learned that I want to communicate is that, one should find the technologies that not only do the job, but have the supporting code and other frameworks that shorten development time (in this case, shorted down from many many many hours to just many many hours )

 

Here are some examples:

 

CodeIgniter for PHP. This helped with forms, validation, application properties, and directory structure. The only place that this bit me was something very specific to this technology, and I will save telling it, so that those who don’t care about learning the specifics of this framework can keep reading.

 

HTTP PECL for PHP made anything http-related turn into easy one-liners in code. Any HTTP_POST, and HTTP_GET that I had to make in Labs 3 and 4 were simple as pie. Also, in Lab 3 I had to parse the response for the submit service, right? I just used HTTP_Parse_Response to take the output from the HTTP_POST to get the “OK” out of there. Awesome.

 

NetBeans 6.0 for Java. I decided that I would do the Lab 5 stand-alone application in Java. At first, I was attracted to C# because I already knew how to manipulate XPaths and XML in there. Also, there is the whole .NET thing to hold my hand throughout the development process. However, the age-old conflict between Microsoft empire and the Open Source world made things difficult. There needed to be some sort of certificate conversion to access AWS, I think. Rather than bother with all that (I did attempt it once, but ditched the effort when I could see it being trouble), I aligned the Open Source technology of Amazon with the Open Source-friendly language of Java. I was familiar with NetBeans 5.0, but had no clue that 6.0 would be that helpful! It had a visual GUI creation environment. In fact, there was a template desktop form with a status bar and menu all set to go. I also found out there was a wizard to add a WSDL. I pointed it to the one given by the lab spec, and it automatically integrated it with the rest of my app, and even put in some basic code to help me interface with it. After working on Lab 5 for about a half a day, I was good.

 

AWS and accompanying classes to help interface. I think because of the fact that it was Amazon, luckily for me, there were already classes written in Java and PHP that interfaced with SQS: Polar Rose in Google Code for Java, and Test Utility for Amazon SQS for PHP. I didn’t even realize at first that I would need something to interface with the SQS, but then I figured it out, installed the classes and some of the dependencies (some required stuff on Apache commons, which place I didn’t even know existed), and finished the coding. Well, now here I am with all my labs done.

 

 

So, this has been quite the journey. Just yesterday, I didn’t even think that I would be able to finish all my labs, because for a long time, I didn’t even know where to start. My head would hurt just knowing there was too much that I didn’t know, and that it would take a chunk of time to figure things out bit by bit. I think my appreciation for the programming community as a whole has grown. I couldn’t imagine trying to figuring things out and THEN write these things from scratch. Just understanding what a solution would look like was resource-consuming enough alone. Implementing the solution from scratch for one lab might have taken a semester, given the coding, testing, and debugging.

 

I would say this is complete, but I have always been wrong in the past, so who knows whether I am actually done.  Chances are, I’m not.

As far as I know, I can call Akismet with no problem.  I kept on using PHP, and the PECL_HTTP library that took me ages to install in the last lab served me very well.  I was able to make http_post_data calls, and to my knowledge everything works….

…except for the fact that I don’t really know how to verify whether my XML makes it to the Processing Queue (http://queue.amazonaws.com/A3N3IV5XJH079S/processing).  Right now, I send a HTTP POST to that endpoint, specifying the content type as text/xml.  If there are some configuration issues, then I should be able to finish them quickly.

Well, it’s done.

I had a surprisingly large amount of trouble parsing XML.  Then I looked a bit more closely at the examples on the PHP docs website, and was able to crank through with it.

I suppose for the latter half of this lab, I really had to focus on “Working Smart” instead of just “Working Hard.” It just seemed like a bunch of mundane tasks (like getting POST to work, or doing an HTTP_POST, or getting XML to parse) were taking up loads of time.

Wow, it was quite the trip trying to get http_get() and http_post() to work in php. Maybe I should have stuck with Python!!

The following sites really helped me configure everything.

http://www.art122-5.net/index.php/How_to_install_PECL_HTTP

http://www.jellyandcustard.com/2006/01/19/installing-pecl-modules/

http://www.linuxforums.org/forum/redhat-fedora-linux-help/58407-yum-install-mythtv-suite-problems.html

http://www.linuxforums.org/forum/linux-applications/41118-how-install-gcc.html

Of course, I’m a Linux Newbie, and so pretty much any Linux site helps me

 

Had to roll back from 5.2.4 to 5.0.4.. I got the PECL_HTTP module on there (http.so), and was finally able to get those dang http_get() and http_post() methods working. Before, there was fatal interpreter error saying it didn’t know what those methods were. (Postmortem comment: crap, I should have just asked John about getting and posting via http… argh.. oh well, this here is one way to do it, I suppose.)

 

Hurrah for Amazon. Wow, having multiple instances of the server versions really helped, though. I would have wasted even more time uninstalling everything to roll back the php version.

Kudos to CodeIgniter. The framework had some nice functionality, that allowed me to do some url rewriting, form validation, templating, and configuration stuff (i.e. setting global variables like URLs).  It was just a light framework that I already knew the basics of.  I’ll have to look more into Smarty though. I have seen many people use it and they have said good things.

Dumped a bunch of time into this lab. Was it worth it? Well, I suppose it’s one of those “I’ll find that out when I’m older”-type of things.

Finally got something to show up in Twitter. Yes, I can proudly declare with the rest of the classmates that have finished Lab 3 that the posting came straight from the application that I wrote…

Still not done, but very close.

Because I want to get this cranked out as soon as I can, I’m afraid that I’ll have to revert, regress, defect, apostatize back to PHP for the implementation. I know of a framework that I can use (CodeIgniter) that will support templating. I already know how to make it so that I can create forms and post, and that sort of thing. I’m just kinda sick of messing with Python and trying to get it to jump through hoops right now. I know that PHP will be able to do the current task just as well. Python, Django, I’ll be back for you… later.

I’m having a bit of trouble getting the POST data (or anything from the request object) in my lab. I tried running this code in my program and in a separate file, but all I get is a segmentation fault (didn’t even know that was possible in Python). I’m pretty sure that the seg fault happens in the loop.

from cgi import escape

from urllib import unquote# The Publisher passes the Request object to the function

def index(req):

   s = """\

<html><head>

<style type="text/css">

td {padding:0.2em 0.5em;border:1px solid black;}

table {border-collapse:collapse;}

</style>

</head><body>

<table cellspacing="0" cellpadding="0">%s</table>

</body></html>

"""

   attribs = ''

# Loop over the Request object attributes

for attrib in dir(req):

      attribs += '<tr><td>%s</td><td>%s</td></tr>'

attribs %= (attrib, escape(unquote(str(req.__getattribute__(attrib)))))

return s % (attribs)

 

Understanding Check:

If I am correct, for this lab we are supposed to basically extend lab1 with the functionalities listed in the lab spec.  I can see the integration process of this lab will be rather interesting.  I realize that I don’t even have a clear understanding of what we are trying to achieve with our web application.  I can’t remember what was said in class.  I can see the individual tasks that the components that we build (from lab to lab) are trying to accomplish, but I don’t understand what the user wants to be able to get out of our application.  How does an “idea” relate to a domain and name, exactly?  I suppose I can get this brief clarification during class today.

 

Issues I have Identified pertaining to this Lab:

Xml Parsing:

I thought about including some library that would support Xpaths, but I think I’m just going to go about parsing the XML the more clunky way of referring to element indices and names.  I assume that the XML to be used as input will be the output from the server we all built in lab 2.

Getting the POST variables:

Well, apparently you can’t use the cgi.FieldStorage() method when using mod_python.publisher.  I’ll have to find some other way to get around that.

Getting the directory listing

The most recent thing that I figured out (with the help of classmates’ posts and Google) was how to modify the content-type header. Everyone knows that if using mod_python.publisher, one must do the following

   def index(req):
     req.content_type = 'xml/html'
     return "<xml />...."

the /domain/site part of the page

I assumed that to display this page, I had to look it up in the listing first, then look up the id, and then grab and parse the json that corresponds to the id, and return the values in the xml structure given in the lab writeup.

I see that I must put multiple <idea> nodes inside an <idealist> node. How would there be more than one idea in an idea list? currently my implementation is that the search for the domain in the listing will stop if there is a match found. I hope I was right in assuming that there would be only one idea or entry returned per domain query.

Was there some outdated information given here, in the page that lists all of the labs? I don’t think there should have been “sitelist” and “siteinfo” services, unless I misunderstood or missed something about the more detailed lab specs.

DistributedLabs

Lab2 – Build the Listing App Server

* Build second server from base
* Build two services: sitelist, siteinfo
* retrieve data from s3 for appropriate service
* translate backend data into result xml
* respond with xml
* register and deregister with load balancer
* Persist server image to S3

I look forward to getting started on lab 3.

After talking with classmates, I found out what my endpoints need to be, and finally figured out what I was supposed to do. I am currently working on the /index method in my index.py, where it lists the domains. There were some server errors returned from running this method, which errors I will have to check out when I get some more time.

The json reader is working, as far as I can tell. I think the next steps I will need to take are the following:

1. Find out what is causing the server error in index()
2. Implement the domain(domainName) function
   1. This will probably require the url rewriting tricks that I have seen classmates write about. Thanks, classmate

After all that is taken care of, and given I am heading in the right direction, I should be finished in another hour or two.