Unweb: A Postmortem

 

In the fall of 2000, I was hip-deep in an addiction to Diablo II. It was the sequel to a multi-million dollar seller a few years previous. At the same time, I was working for a web development firm in Denver, Colorado, called Breckenridge Communicatioins. At this company, my role was that of Senior Programmer, and my responsibilities were to fill the role of the ColdFusion & JavaScript developer. Through a series of mishaps, I blew a CPU and Motherboard, and lacked the necessary funds to replace them in a timely fashion. Since I had to resort to using a lowly Pentium II 300 in the meantime, I had to satisfy my Diablo II hunger in some other fashion.

In the fall of 2000, I was hip-deep in an addiction to Diablo II. It was the sequel to a multi-million dollar seller a few years previous. Since I hadn't played the first, the second became that much more intriguing to me. At the same time, I was working for a web development firm in Denver, Colorado, called Breckenridge Communicatioins. At this company, my role was that of Senior Programmer, and my responsibilities were to fill the role of the ColdFusion & JavaScript developer. Through a series of mishaps, I blew a CPU and Motherboard, and lacked the necessary funds to replace them in a timely fashion. Since I had to resort to using a lowly Pentium II 300 in the meantime, I had to satisfy my Diablo II hunger in some other fashion.

With some time between projects, I began to experiment with the <CFHTTP> tag, which allows ColdFusion to make an internal http request to a web server and process the results as a variable. I wondered how easy it would be to parse the data that was returned from an http request and generate loose "statistics" off what was found, such as font tags instead of cascading style sheets, or tables that weren't properly closed. In a sense, I attempted to see if I could discover errors in other people's HTML.

My initial tests proved successful, and I was able to generate basic statistics from these parses. In order to keep my interest, however, I had to make the project fun, so I turned it into a game. The idea was to award points based on good html, and to take points away, if bad html was discovered. When the score was tallied, the web site was given a rank. At that point, I decided it might be fun to use this rank as a "health" or HP value. I then began playing with various other factors and came up with an Attack Rating and Defense Rating for the website (how hard it attacked, and how well it defended). One thing lead to another and I soon had a player concept as well, allowing a person to type in a URL, parse the HTML, and fight the "monster", with a series of clicks to a "Fight" submit button.

Thus, Unweb was born. The game grew and went through many evolutionary revisions, adding support for multiple items, new item types, ladder rankings, and more. Inevitably, Unweb sufferred through several moves to various webservers itself, where a combination of poor programming and detatched administration led to its eventual downfall. For as long as Unweb was alive, however, it seemed to offer a bit of a distraction in the form of some fun and vengeful tactics for the wary adventurer who had a few minutes to kill, and a website that needed to be beaten on.

What Went Right

In developing Unweb into a full-fledged webgame, I think that I had a number of ideas that turned out rather well in the finished game. Whether or not this was a result of careful planning, cunning ingenuity, or pure dumb luck has yet to be determined.

  1. Parsing Meta-Tags for Random Item Names: One of my earliest ideas when I began modelling Unweb around the whole "Diablo" theme was to have items dynamically generated after a website was defeated. Parsing Meta-Tags out of HTML can often reveal interesting tidbits about sites that are commonly used by Search Engines as index data. My thought was to parse these keywords out and turn them into the names of the items, to add a bit of flavor, humor, and randomness to the game. I figured that if a player tried Unweb out on his or her own website, and was awarded a weapon that a family name, or some other piece of random info it it, the player would be amused enough to try again. This seemed to be a successful tactic, since players responded as finding that one of the most interesting (and funny) aspects to the game.
  2. Allowing Players to Sell Items To Each Other: When Unweb went into early beta, I approached a few programmers at Macromedia (then Allaire) about their opinions on the game, whether or not they thought it was fun, and how I could improve any of the CFML code behind it. One suggestion came in from Raymond Camden, author of the famous "Death Clock". Since items were dropping in the game, as well as gold, he argued why I wasn't supporting the ability for users to sell items to one another. My initial reaction was, how on earth would someone build support for two users selling items to each other, that aren't necessarily online at the same time. Ray responded with simply, "Allow the player to put the items up for sale, as a fixed price, and allow other players to browse the for-sale items at their leisure, picking up items that they want (if they have the gold)." Duh! It couldn't have been simpler. Adding in support for items being sold back and forth between players proved massively successful, especially with "collection factor" of the set items.
  3. Implementing a Diablo II-style Ladder: Ignoring the fact that Unweb was a game, it was at heart, simply a website, which meant many users would be accessing it at once. So, in order to give the players a sense of "purpose" to the game, I opted to add the competitive factor of a Ladder, much like what is used in Diablo's Battle.Net service. This also proved to be a great addition to the game, since it not only encouraged users to continue to play more (to vie for the top-most position), it caused them to bring in their friends, who in turn invited more and more people, in order to compete.
  4. Optimizing Logic in <CFSCRIPT>: The brunt of the game logic fell within each attack round. This is where all the bonuses, items, armor, special attacks and more were thrown together into a damage calculation and applied to the website being hit, and the player doing the attacking. In order to get the absolute maximum amount of performance out of CFML, I ran through several iterations of optimizing the code, which inevitably led to doing all the calculations in <CFSCRIPT> tags. Moving from my initial code to the optimzed cfscript shaved several seconds the final product, which ultimately all adds up when hundreds of users are hitting the site.
  5. Moving the HTTP Fetch/Parse to a COM Object: It became clear early on that <CFHTTP> was not going to be robust enough to handle hit after hit in a game of Unweb. Thanks to a fellow friend and programmer, Dave Sausville, I decided to move the logic of the HTTP fetching and parsing of the web result into an externally compiled C++ DLL, in the form of a COM object. As you can imagine, this improved the speed and stabiliy of Unweb immensely, and was probably the single best decision throughout Unweb's development.
  6. "Pimping" the Website out to Diablo Fans: Once Unweb finally went live for its initial run, I immediately began emailing website admins about it. The goal was to get the word out and hope that it would spread, based off the user experience. The most successful clique of this fanbase ended up being fans of the original Diablo II game. I made sure to contact the admins of planetdiablo.com and diabloii.net, and as well, hung out in #diabloii on IRC in order to drudge up some more users. This, in my mind, was a success, simply because they went in understanding the basics of the game, and getting the "inside humor" that gamers not familiar with Diablo missed. It was probably their word-of-mouth that led to Unweb's userbase eventually growing to 15,000+ users.

What Went Wrong

  1. Putting Too Much Faith in <CFHTTP>: Since I was initially only "playing" with the idea of posting out to a user-specified website and parsing the result, <CFHTTP> seemed to be the most natural solution (since I was using CFML). However, I should have trusted my gut instincts more and decided on using a different technology sooner, since it is a well-known fact in ColdFusion circles that <CFHTTP> isn't dependible and doesn't do well under load. Alas, it was the first giant bottleneck of Unweb when the game went live, and cause the servers that hosted it to buckle and completely shut down at times. I did eventually move to an external COM object (see above), but in all honestly, I should have avoided <CFHTTP> like the plague from the very beginning.
  2. Overestimating CPU Usage: Unweb was a very CPU-itensive web application. In retrospect, I really overestimated what the web server could handle when it boiled down to sheer number crunching, which is exactly what Unweb was, at heart. Everytime a player clicked the "attack" button, the calculation of all the bonuses, all the damage, all the defense and so on, had to be re-crunched. Sounds somewhat pitiful, when you think about it (isn't number-crunching what programs are supposed to do?), but Unweb was written in CFML, which is a scripting language for web applications. Most games, however, benefit from compiled binary objects, that are optimized for speed. Since I had never implemented a game in this fashion before, I really had no point-of-reference to gauge what kind of performance I would get. Alas, the real test came when Unweb got beaten on by players, and if I knew then what I knew now, I wouldn't have written Unweb in CFML at all.
  3. Underestimating Cheaters
  4. : Point 3 kind of goes hand-in-hand with point 2, simply because I knew that with many online games come cheaters. However, I honestly had no idea Unweb would become as popular as it did, so I was never thinking about cheaters as having a significant impact on anything. If I found out someone was cheating, "big deal", I thought. So they get to the top of the ladder in one day, or figure out how to get a specific item to drop, or maybe to decode the "randomness" of the gambling. All of these issues paled in comparsion to what cheating players ended up doing, which was causing the actual server to buckle. They did this by writing "scripts" which automatically played Unweb for them; logging in, selecting a website to fight, and the actual clicking of the "attack" button...all done with some clever code. As clever as it was, these "scripts" acted more like load-testing tools, smashing the server with hundreds upon thousands of "hits" every minute. I managed to squeeze in some last-minute anti-cheating ploys, but to this day, I still get the occaisional email from someone who managed to get around them. Note to self: If you make an online game... factor cheating in. Period.

  5. Ignoring SQL Server Indexes: This is your typical slap-me-I'm-stupid reason. There is absolutely no excuse for not having indexes on a database. Unless, of course, you're a moron and forget, as was the case with Unweb. With focusing some much on the actual fight algorith, creating random item drops, building a store, supporting a ladder, and so on...I forgot one of the most basic optimizations every database should have. For those of you unfamiliar with Database design, an "Index" is much like an index in a book; it allows the database to "look up and jump to" specific data very quickly, as opposed to scanning page after page after page of data until the required info is found. All good DBAs will tell you a properly designed database will be indexed appropriately. On Unweb's release day, its database wasn't. That's a bad thing. And yes, I have been spanked for my behavior.
  6. Botching the Authentication Process: Let's be frank: I specialize in back-end application logic development. I almost never touch the UI (user-interface), save to drop a few form fields into a webpage (and I prefer it that way). Most of the UI work done at my office is done by a team of experts that specialize in creating and optimizing UIs to suit the needs of our various clients. So, when it came time for me to put a front-end on Unweb to allow players to actually "play" (who would've thought!!), I missed the boat big-time. For starters, if you want to entice anonymous people to your site, to "see what all the commotion is about", the first thing you should NOT present them with is a login/password window. I would have to say 75% of my initial traffice started...and ended...on the login page. Creating a new account should have been much more intuitive, and anonymous users should've been allowed in for a few "free" rounds, first.
  7. Not Having a Dedicated Webserver: About the time that Unweb was getting ready to go live for the first time, I had a number of options for hosting. I had administrative control of a dual PII-400 w/256 mb of RAM that was already acting as a webserver, with ColdFusion 5 installed. However, it was also being shared by a group of friends to do some "online gaming" in the evenings. Once I saw how the initial load of users spiked the processors (see point 2), I knew that idea was out. I had a friend running a webserver on a Duron 700 w/768 mb of RAM, so in theory, that made a bit more sense. Theory, unfortuantely, isn't always close to reality. That box was constantly coming down after Unweb was hosted there, no doubt related to point 1. After a short stint there, I was approached by a fairly large online gaming conglomerate that offered to host my site for free on one of their uber-boxes, a dual PIII-900 w/ 1GB of ram and a "fat pipe" (about a T3) worth of bandwidth. However, they backed out, once they discovered the game was written in ColdFusion, and since they were mainly an ASP house, weren't willing to fork out for the hefty price tag of a CF license. Finally, I moved the site to an inexpensive CF-geared host down in Texas. Unweb has the least amount of problems here, but alas...it was still causing CFServer to crash from time-to-time, which is simply not acceptible on a production host. Since Unweb was moved from home to home more than an orphaned child, it was doomed from the get-go.
  8. Not Implementing a Caching Mechanism for Users: One of the best optimizations that Unweb never got was a "timed" commit of all user-specific vars to the database. Originally, Unweb would commit all the user's data to the database at the end of every fight. Well, some fights could last mere fractions of a second (nevermind point 3). All these extra commits to the database could've really lightened the load of Unweb, if they had been simply stored their data in memory, and commited changes-only after a given time period. Coincidentally, I began to design a "sequel" to Unweb that subsequently fell to the wayside, but one of its features was exactly this: a timed mechanism for commiting data. The performance really was improved and I honestly believe that Unweb could have benefitted greatly from this simply optimization.

In Conclusion

All in all, Unweb was an exciting adventure to undertake. I have built small games in the past (and I intend to build more in the future) but I think Unweb struct a chord with many players. Not only was it a total rip-off of Diablo's style, but the idea of fighting websites was a real "kick" and I truly believe players that hit it hard, did so because they got some amusement out of it. I still receive emails on a regular basis on "whatever happened to that web-site fighting thing?" and "is Unweb hosted anywhere?" (I even get requests for source code!), and all of these I appreciate, since I am still amazed at the response it garnished. The best response I think I ever receieved was from a professional game-developer, who considered me one for Unweb's creation. To that, I am truly honored. Perhaps one day in the future Unweb may live again in some other form, but for now, Unweb has fought its last website...

...or has it?

I played Unweb a long time ago, and I had a blast with it. But for one reason or another I didn't have time to play it more often. Condolences on the apparent demise.

Smacking around Britneyspears.com and Microsoft was a fond memory.

When Unweb was pulled off the 'net, the top 5 players were:

# 1. Alexander the Great - 179,613,304,687
# 2. KevinG - 150,000,167,817
# 3. Than - 134,722,899,513
# 4. Yggdrasil - 132,316,171,201
# 5. Anthony - 106,293,849,971

As expected, they were all lvl 99.

I don't remember any of those names, I remember that for some reason, a short while before Unweb was gone, the ladder I was on was reset or dissappeared. I think my name may have included the word "mig" somewhere in there. I also was not on the Hardcore ladder, if that helps at all. I remember that I was fairly close to #3 but the #1 guy had a huge difference in his score. I'm not sure what my level was.

I played it yesterday. Now the server has imploded. It's annoying.

I shut it down. It wasn't for public consumption.

fate hates me
i find out about unweb, and am imidiatly enticed. i play one round and am forced of the computer
i come back the next day and, viola! unweb is gone

Any chance Unweb will come back to life soon? Saw link on Reallifecomics.com and am very interested.

I also tried it from the reallifecomics.com link, and I really liked it!!!!! Here is hoping you will revive it in a more stable and more efficient form!

I saw the link at reallifecomics.com, too, but by the time I arrived it was gone. I'd like to see it, I hope you revive it soon! (That link doesn't work, either.)

I'd like to see the link on reallifecomics.com, if anyone has the specific url to the mention of Unweb.

It is to the left at the start page of http://www.reallifecomics.com under "Cool Link of the Week"

Yes, RLC too. It seemed so cool... Like TKEP, but better. Oh well. I guess I should go back to TKEP anyway. It still has my soul...

just a note: Yggdrasil is
a) a huge tree (tree of life?) from Norse (?) mythology
b) a computer in the popular japanese manga/anime Ah! Megami-sama
It's not a hard proof, but sounds a lot like behind this account there is actually a script
...just a thought ;P

I found out about Unweb from Reallifecomics.com too, just now. Tust me to miss such a ... Cant think off a word to do it justice idea. I really hope you get Unweb upp again, id pay to fight microsoft or crush apple under my heal.
One question; Did you implement grouping in that? Takeing down hardened monsters is soo much more satisfying if you bring your friends...

Yggdrasil, if its the name for the tree in Norse mythology (i honestly cant remember) would be the world tree, as in the planet\world. Just incase anyones interested.

The only real impact it has on the numerouse god sagas is in the tail of the Norse armagedon, the Ragnarok, where 2 humans hide in its branches and so survive the final battle between the gods and the trolls.
Something im sure bush will bring about any day now.
Hanzo, WRITE MORE GAMES! NOW!

I never did implement parties or even pvp (one of the most requested items) but I did spend a bit of time on a sequel which implemented a drag-and-drop inventory screen a la Diablo II, and also utilized a kind of psuedo-real time partying fight system...kind of like Final Fantasy.

These of course all fell to the wayside...the concepts are still fresh though, but I am loathe to revisit any kind of web-game development at this point. I would much rather deliver a *real* game...

How about seeing if anyone else is willing to take it on and continue updating/fixing it?

Im sure there would be someone out there with the required skills to keep it operating.

I have definitely put out the request and if people are interested (some interest has been shown already) perhaps it'll get another life.

One of the best thing to do, maybe, would be to put in on Sourceforge. That way you can get developers to work on it, and you can get a server on which to test it (though I doubt the SF admins would be too happy to have it *run* off their servers for the general public. If you're interested, my sourceforge devloper name is zxaos.

I think part of the manic appeal of unweb was its creativity and a way of allowing people with little knowledege of programming to "strike back" at the things they saw as big, stupid corporations, or bad design. Power to the people and all.

I know full well that I could be retarded in suggesting this as I am SURE it has already been thought of - but would there be any way of releasing like a local client that would do all the crunching and run completely locally independent of a central server (ya I realize this opens the door for super cheating and no ladders etc, but it would give the chance to all of us who just loved the game to be able to play it for ourselves :)
Just an idea,
Thanks.

WHO DID PLAY JULIE THE CAT GAFNEY IN THE MIGHTY DUCKS IF U WANNA PLAY LIKE HER

If any of you are interested what I have been up to as of late, you can read an interview with my partner in crime, Joe Mease, here:

http://www.open-online.org/interviews/joem/

As of late I have been embracing more Flash MX, specifically as it pertains to XMLSocket and the transfer of data in a client-server environment (instead of the pseudo realtime stuff we did with the GDX referenced in the above article). More than likely I'll be working with Joe on some kind of multiplayer game in Flash at some point in the near future.

Bring it back :D