Nathan at Work

• setting up twitter… #
• uploading my aunts head.. #
• changing colors… #

Powered by Twitter Tools.

here’s what I really want to be blogging about, what I really want to be addressing, and what niggles at me every day of life.

The internet is broken, or perhaps more accurate to say the technologies that form the major part of the internet are massively misused, infact by design and spec they don’t support the internet as we use it, the internet we are used to.

By internet I mean “the web”, and by the web I mean the same web that’s in website, webdesign and web development, you know everything you ever see in a web browser.

Let’s start at the core of the internet, the main protocol HTTP/1.1 and see how it’s misused.

HTTP/1.1
It’s described perfectly in the Abstract of rfc2616 (the HyperText Transfer Protocol).

“It is a generic, stateless, protocol”

Straight away we have a major design flaw feature that points to how we misuse this protocol. HTTP/1.1 is a generic (that’s good), stateless (virtually every web application and website you see is stateful, hows that supposed to work over a stateless), protocol.

So how does it work.. from the section 1.4 of the rfc which describes the Overall Operation of HTTP/1.1 we get this:

The HTTP protocol is a request/response protocol. A client sends a request to the server in the form of a request method, URI, and protocol version, followed by a MIME-like message containing request modifiers, client information, and possible body content over a connection with a server. The server responds with a status line, including the message’s protocol version and a success or error code, followed by a MIME-like message containing server information, entity metainformation, and possible entity-body content.

Simple.. a web browser sends a request packet to a web server (normally for a resource) and the webserver responds with a response packet containing said resource.

Let’s run a quick example.. say you visit facebook in you’re web browsers, here’s what happens:

1. Browser sends a request to the web server hosting the domain facebook.com
2. Facebook.com web server responds with the markup for the default page in it’s present state.
3. connection is closed
4. web browser parses the markup in order to display it and to find any related resources it may need to request.
5. for each resource needed (image, script, advert, stylesheet, icon..) the web browser..
   1. sends a request for the resource
   2. recieves a response hopefully containing the resource
   3. closes the connection
6. for each request (from above) recieved by the web server it
   1. allocates or creates a thread/process for the request
   2. thread/process resolves the response, locates the correct resource and parses it
   3. thread/process returns a response including the resource
   4. thread closes or web server deallocates the process

Even on a light page with only 10 images, 2 stylesheets and 2 javascripts, that’s 15 requests and responses, each one a seperate connection to a seperate process on the server. Slow, time consuming and a lot of overhead for something so simple.

HTTP/1.1 brought some new features to address these things, such as the connection header; primarily used to say when a connection should be close (connection: close); it allows a client to send in say 20 requests and recieve 20 responses in the order they where requested; thus only using one connection and one process. Is this used.. no; infact most webservers simply send back connection close and close the connection even if you don’t want to.

Further connection close is a bit useless in the scenario above, to be implemented correctly the webserver would need to parse the local resource, identify all resources it held itself, then send down responses for those as well; this however would break the protocol as no request was sent, and also it raises the question “what if the client doesn’t want the images or scripts, or has them cache’d?”.

Another vital limitiation of the Request/Response protocol; there is no allowance for a server push or on the fly update, there’s no way for a server to send anything to a client without the client first requesting it; a massive short-coming that leads to some of the worst misuse of HTTP/1.1.

Consider, a simple shoutbox on a small website, 10 people chatting in it concurrently. There are two ways to misuse the protocol to enable this.

1. Polling
   A small javascript or meta refresh runs in the web browser; every second it asks the server for the entire contents of the shoutbox page again, or through further misuse of another technology (javascript), asks the web server “any updates”; continually, like a nagging child, once per second, for every user (10 users = 10 requests per second to the webserver only for shoutbox.. that’s a lot, 10 requests per second on a normal website = 864000 page views a day / circa half a million visits..)
2. …

I’ll finish this post if warrented tbc?

So Maven in eclipse is a great time saver (and waster); in that it wastes a tonne of time setting up and learning, but saves a tonne of time in the long run on team projects.

This weeks maven problems have been running suites of testng test when doing a maven build; the old pergen memory not set high enough faults; but this time setting the paremeters was a bit harder.

here’s the fix(es):

set pergen space for eclipse (3.4) higher:
- run eclipse from the command line with the options:

eclipse -vmargs -Xms1536m -Xmx1536m -XX:PermSize=1024m -XX:MaxPermSize=2048m


- give maven more memory

export MAVEN_OPTS="-Xms1536m -Xmx1536m -XX:PermSize=1024m -XX:MaxPermSize=2048m"


now the bit that took ages to figure out – the maven eclipse plugin runs testng test through surefire on the console and returns back output to eclipse, not only that but it’s run through a jar using a jre – so to give this instance of the jre more memory you need to:

1. Right Click ProjectName > Run As > Maven Build…
2. Goals: (one of install, deploy etc)
   install -e
3. JRE Tab > VM Arguments:
   -Xms1536m -Xmx1536m -XX:PermSize=1024m -XX:MaxPermSize=2048m

AFAIK this and only this will allocate enough memory for the the test suite to run

Joy

These are more for my own reference, but may well help any readers as well, specifically they relate to using maven with the maven plugin for eclipse.

Maven “Package”
Will deploy the application to you’re local target directory only.

Maven “Install”
Will deploy the application to you’re local target directory AND to you’re local .m2/repository

Ignoring test failures when building
Project -> Right Click -> Run As -> Maven Build…
“Goals” => package -Dmaven.test.failure.ignore=true
>Run
(swap package for you’re goal)

Skipping tests all together when building.
Project -> Right Click -> Run As -> Maven Build…
“Goals” => package -Dmaven.test.skip=true
>Run
(swap package for you’re goal)

Additional non java files are not deployed (needed properties, xml files, beans etc)
Project -> Right Click -> Run As -> Maven Clean
Eclipse File Menu -> Project -> Clean
then build again and all should be fixed.

I’ll add more to the list as I find them.

When deploying applications using apache maven you often run in to dependency issues, two conflicting jars and the wrong one being used.

I recently ran in to this one when trying to deploy a multi-project application we are building at kraya; basically hibernate and cxf both depend on slf4j-* jars; hibernate down in the domain model project was depending on a newer version and using the method org.slf4j.impl.JDK14LoggerAdapter.trace, however the final web app depended on apache cxf which used an older version 1.3.1 of slf4j, which didn’t have the trace method.

Ultimately this led to an exception when deploying, “java.lang.AbstractMethodError: org.slf4j.impl.JDK14LoggerAdapter.trace”

To fix was a case of going through the dependency hierarchy in the web apps pom, locating the conflict on slf4j then changing the pom’s to make everything depend on the newer version.

from the maven site:

Dependency mediation – this determines what version of a dependency will be used when multiple versions of an artifact are encountered. Currently, Maven 2.0 only supports using the “nearest definition” which means that it will use the version of the closest dependency to your project in the tree of dependencies.

Translated this means the higher a dependency is in the dependencies block of the pom, the more weight it is given. In our case the resolution was to move the DomainModel project (with the newer hibernate slf4j jar) to the top of the dependencies. This fixed one of the two conflicts, namely slf4j-api.

Next up was to go to the pom of another dependent project (in the pom above cxf) and add in the dependency for the latest 1.5.6 same 1.5.2 version of slf4j-jdk14 as hibernate.

Job done, and no old 1.3.1 slf4j get deployed; thus no exception.

joy, and simpler than I though.

It seems that my coworkers, even my boss have gone a little bit soft today, in a very unexpected but reassuring way.

While considering the beauty and love in life, both chris and shri have ended up reverting the subject back to code, even bringing it right back to the same coding subject; one which was ultimately on my mind most of the day and last night, infact it’s always been there and very much defines why I am a coder.

On the subject of love, and more specifically how to deal with a certain woman, chris was debating two different approaches. The approach he took, and has always taken, was the right one I’d say.. simply because the reason a relationship ever get’s to a certain point, is because of the way both people are, to suddenly change the way one of you is and do something different can’t possibly work; it’s breaking one of the foundations on which the relationship was built.

Anyways, as developers do chris brought this back to the fact that there’s often two or more approaches to most problems, then went code specific; the most common way being to wade right in and wrestle the code in to submission, get it to work perhaps without understanding why, and assume or hope that it will continue to function that way.

The other approach chris pointed out, is to take it slowly, take the time to fully understand the problem, read, study, learn and finally work out the root cause of the problem, taking the correct path to write a beautiful, elegant and stable solution.

Meanwhile shri.. well there’s no way of putting it other than to quote..

I invite you to think about this problem as a rose… a once in a lifetime chance to appreciate something beautiful. Don’t just hammer away at it, break out of your habits, out of your box, destroy your preconceived ideas and look at it differently, try something new…

At the very least, you will learn something new and you might appreciate the issues surrounding it better.

Why so reassuring? well it points to them both being problem solvers, not only that but also appreciating the chance to solve a problem, studying, finding out all about it, fully understanding the task at hand before carefully implementing the correct solution to the problem. You’d be amazed how many coders are not problem solvers, lacking the ability to think on their feet and discuss things at a conceptual level, often even finding and fixing problems before they have happened.

Perhaps vainly I see myself in the problem solving group of coders, I’ve always thrived on a good problem, specifically code wise, a broken website, a server that just won’t play nice, a task that seems impossible, a problem with technologies I have no prior knowledge of, even a task that nobody else dare take on; to me that’s gold dust and the reason why I do what I do. I love a good solution, and can find beauty in well written block of code. Love to learn, and love to try new things, infact I’d never met anybody who tried new (everything) as much as me until I met shri, tis great!

To know the people at the helm (as it were) are of a similer mentality is massively reassuring; Primarily because every post I write, every peice of code I do, every job I’ve done for many years, my whole career even is pointed at analysing a single big problem, and finding the solution (which I think I have..). There will be a post soon about it, called The Broken Internet, it’s going to take me some time to finish writing as it’s a few years of research that needs to be made in to something easy to read; but it’s coming.. on the same note, the series of posts I’m doing on designing and developing this blog all lead to the same point, so I may as well skip to the end first.

@note
Kind of hoping that once Shri and Chris, and hopefully more of my kraya coworkers have read up on what I’m talking about, it’ll at best spur them on to help me implement or at least plan the solution, at worst I’m sure it’ll just niggle away at the back of there brains (as it does with me) and shape the decisions they make.

so ominous.

@context
I started this post at 1400, finished it around midnight; between I spent the day lugging furniture around the house; we had a problem you see, the kids basically overrulling the main floor of the house, we thought about it long and hard (Rachel and I) and eventually it came to light that the problem was because one of the kids bedrooms was on this floor, as was their toy room/living room. The solution was today, to move all the children on to the upper floor and remove the toy room; moving our bedroom on to the main floor of the house and thus making an adult only space in the house. Rather than kids running around us all day, we now have footsteps above, a great deal better. This ties in well though, as to get this fix I had to sacrifice my nice big bedroom with en-suite (not best pleased about that) but the simple feeling of freedom and space in the home is without a doublt worth it. So.. the correct solution isn’t always one you’d prefer, but to fully solve a problem you have to go for what’s right, not what’s easiest.

Earlier I posted about picking a font for this blog, which got shri to thinking further about fonts.. in turn I thought a bit further and perhaps realised something(s)

• in author/designer mode I want to further express myself via the use of font
• in reader mode I would prefer to have some standardised format for reading everything
• depending on the mood, and importance of how quickly I need to retrieve a bit of information, I may also want to see the content as the author/designer intended
• it would be useful if my choice of display font was portable
• meanwhile it would also be useful if a standard use anywhere font choice was globally accepted, for instance what font would be used on the instructions which tell you how to setup you’re roaming font profile? or on the os installation screen?
• if I’m reading in arabic though that english font would be a bit useless
• I’d hope the font would support all character encodings and be supported by all applications
• choice of font is context specific, I want to read code in a different font to a general post, and I’d prefer a title to be in a different font to the paragraph text.. and shri mentioned it’s a decision at the semantic level.

I believe it’s impossible to address in full, or even part; the above issues will be ultimately resolved when we have the perfect invisible interface, true HCI, the age when these words are simply thoughts with context attached and semantic information that’s injected directly in to ones brain.

@future-blog-maybe
All of the words I’ve ever written, and indeed anybody, have been written to convey some form of thought from one human to another; what we really need is a method of transporting thought between humans; you’d never misunderstand, mistake somebodies tone, take something out of context and you’d be able to understand and indeed learn, instantly and efficiently. Until that day we’re all lumbered with spending most of our time trying to convey a thought to other people, and paying for the privilege dearly (every script or program I’ve ever written was to in some way address this conveying of thought issue).

I couldn’t stand that damn Kubrick default theme any more so as a temporary measure I enabled one of the default themes to appease my soul / eyes.

phew

@todo
ensure this is a temporary measure

@tangent from /blog-design/font-frustrations
I can’t recommend VirtualBox enough; all my life I’ve found frustration in needing tools from both Windows and Linux, which led to me constantly keeping that putty client connected to a linux server. Not now though, no no no no (as two unlimited said);

Here’s a very quick outline which should convey why I’m so pleased with VirtualBox:

I have a full Ubuntu Server 8.10 with the desktop installed running in a virtual box on my vista ultimate, not fullscreen but in a window 1360*768 (nice aspect ratio) which I can easily just alt-tab to and control, if I want to use the mouse I just click in and use, to get the mouse back out I tap the right-ctrl.

Files.. using samba I’ve got all my windows drives mounted in /mnt on ubuntu so common files throughout.

Network.. currently I have a bridge set-up between the virtual box connection and my windows connection, with a bit of tweaking this has allowed me to give the virtual ubuntu it’s own ip address from the router. Thus on my network I have my desktop, the virtual ubuntu, the tablet and rach’s pc. Additionally with some config of port-forwarding I’ve got ssh, mysql, maven proxy, tomcat and apache (port 81) all running on ubuntu and exposed to the internet via a domain mapped to my static ip, whilst every other port is likewise exposed and set to the vista desktop.
@note planning on giving the virtual box it’s own gigabit network card, hence the “currently”.

It’s so liberating because.. well consider, got some files on my desktop (literally on the desktop of my vista) at home that I could do with at work:

• Open terminal
• scp -r nath@mydomain:/vista/Users/Nath/Desktop ./
• done.. and I’ve got those files at work.

The other way.. Shri and Chris have been working late at the office and released a new snapshot into the work-office-local-network maven proxy on shri’s machine, I need them in my maven proxy at home as I’m also working late.. a few weeks ago that was it, can’t work, now though

• alt-tab into ubuntu
• ssh in to the work server
• ssh over to shri’s machine on the private office network
• cd into his .m2/repository
• scp -r ./uk nath@mydomain:/usr/local/maven-proxy/local-repo
• exit exit
• alt-tab into eclipse running back on the vista desktop
• right-click project, maven > update snapshots
• done – all working!

Gripes:
Only one.. the copy paste bucket isn’t shared, so often I copy in windows and paste into ubuntu to find it’s not there. I think we can say thats minor all things considered.

Go get Sun xVM VirtualBox now.

Last night I spent a wedge of time considering the various forms of container I could choose for this blog, I then moved on to fonts, which I’ll cover in a minute.

@context
I’m feeling a bit frustrated today and trying not to loose interest, I’m one who likes to get my head stuck in to something and not stop until it’s complete. The approach of considering every single decision made whilst designing and developing this blog is therefore doing my nut in a little; alas though I feel like I should stick with it and cover all the incredibly boring subjects which have plagued my life, work life anyways, for many years. On with the show then.

@research @resource
seeing as I’m going to be using xhtml strict along with css 2.1 (maybe 3..), best to find out what w3c has to say about the css font-family property in short they recommend going for a font-family such as helvetica followed by a generic font family such as sans-serif.

Perhaps the most basic choice of all, which font family to use on the blog, an easy choice one would think – just skip through the font list and stop at one you like. I wish.. as with all things web you need to consider support, somebody could be viewing this site on a blackberry, playstaton 3, any of the web browsers, on any of the operating systems, with any amount of standard or non standard font’s installed.

Realistically then I need to choose a font family that is common to at least all operating systems and web browsers from mac os x to ubuntu via windows. fun.. but first I need that list.

@research @resource
After much googleing (-e?) I found this handy web font blog post by Dustin Brewer which gives me more or less that list; coupled with these additional dated yet useful notes from Browser News I do believe I’ve got my font armory all figured out.

The Choices.. playing it safe I can either simply select a generic font family or one of the following combinations:

see how short that list was? I thought it best to double check, so I quickly made a list of all the fonts on my windows vista ultimate box, then opened up the latest version of Ubuntu which I’ve got running in the fantastic and liberating xVM VirtualBox from Sun. Guess what, other than monospace there were no common fonts between the two; a quick thought later and it seems obvious that there will be zero common fonts accross all platforms, hence I’m sticking to a generic font family.

@tangent on virtualbox worthy of it’s own post

@context
writing that post on something I’m interested in and have postive thoughts about was a nice break from this boring but needed font post, consider spirits to be listed a bit.

One could simply select a font family on each os, put them in order in a font-family css property, but that would require picking a font on each os, checking back to see it’s on the last few os versions and then hoping for the best; browsers aren’t actually required to honour the designers choice of font you see. (I don’t have the time, the hundreds of machines or wish to do so, if you do.. knock yourself out and good luck!)

The choices:

• serif
• sans-serif
• cursive
• fantasy
• monospace

as BrowserNews said.. “a paltry choice”.. actually it’s worth checking this list to see what they look like on your machine.

After checking on both Ubuntu and Vista, I’m going with:

• text: sans-serif
• titles: serif
• code: monospace

I am very tempted to go with tried and tested font-family combo’s though:

• text: Arial, Helvetica, Univers, ‘Nimbus Sans L’, Tahoma, sans-serif;
• titles: Georgia, ‘Times New Roman’, Times, serif;
• code: ‘Dark Courier’, ‘Courier New’, Courier, monospace;

Hard for a blog to be unique, well designed, smary with such lame choices, somebody should address this.

@context
considering how crap life would be if everything could either be black, grey, brown, green or white; although they are possibly the most common. Saturday night and blogging about which font to choose while the other half updates the mp3 site and fills my ears with next years news though, so not all bad.

@tangent
I always want to finish every post “regards” like it’s an email or a mailing list post.

regards