The cake is a lie

Misconceptioned Memcached

The magical memcached layer

You’ve heard of memcached, haven’t you?

Made by Livejournal to sustain millions of angst-ridden teen diaries, now used by web giants like Slashdot, Digg, Wikipedia, Facebook, Youtube, and countless more. Even this very Wordpress blog supports it.

The same misconception seems to haunt all those unfamiliar with the memcached: it’s some sort of magical layer that sits between your database and your application layer, and somehow it makes things faster. The magic is scary. How would something know what it needs to speed up from the database? It must require all sorts of nasty configuration, all sorts of nasty corner cases to cover?

Admit it. The thought of using memcached terrifies you. Well, I have good news for you: That’s not what memcached is. Are you read for it?

Memcached is a hash table.

No, seriously. It’s that simple. It’s a hash table that sits in memory, that can scale between multiple servers. All you do is launch the memcached daemon on your server with memcachedand you can use it!

Well obviously that’s not all there is…” you say. It is! As far as the server is concerned, that’s all the configuration it needs.

Now, the client side…

How does the hash table work? It’s just like a dictionary. To save data: you send it a key/value pair, and it saves it. To retrieve data: you send it a key, and it gives you the value. Explicit. Simple. Efficient. It supports a handful of simple operations like set, get, and replace.

Here’s how to use 3 distributed memcached servers to cache a huge query in Python:

import memcache
...
cache = memcache.Client(['192.168.1.100:11211',
                         '192.168.1.101:11211',
                         '192.168.1.102:11211'])
...
def get_post(post_id):
    "Return data associated with post with given post_id"
    result = cache.get(post_id)
    if not result:
        # No cache, perform potentially costly SQL query
        result = model.Post.get_post(post_id)
        # Save result in cache for next time
        cache.set(post_id, result)
    return result

Do you see what they did there?

Yes, it’s not magic. Yes, you have to cache things explicitly. Yes, this is incredibly powerful. Now go. Go make a Slashdotting-proof web service, that’ll show them.

1 comment

Gentoo, Rockband, Code, and Music

Today, you’ll get to hear about what I’ve been up to! This wont be on the exam.

  • After a year and a half stint with Ubuntu, I’m back to my true love Gentoo. That is, with a shiny matte new quad core beast of a machine.
  • Bought Rockband for my PS3 couple of weeks ago. Drums are ridiculously hard. Been rocking out. Good fun is being had by all those who dare rock. Can’t wait for Still Alive to be released for free. It shall be a triumph!
  • What started out as pretty code is now a bonafide open source Python module: workerpool. People are using it. No, really.
  • muxtape.com: A super simple music sharing web app launched last week. Its been enriching my life — doing what Pandora once did. Here’s my muxtape. Be right back, there’s someone at the door.
5 comments

Blu-ray, scientology, and J. J. Abrams

  • Sony’s Blu-ray beat out Microsoft’s HD-DVD in the hi-def wars. Sony stock is going up. Microsoft stock not so much, especially with the recent bid at Yahoo.
  • Scientology is now censoring eBay, as its has been permitted to arbitrarily delete auctions. Boycott time?
  • J.J. Abrams, an ex-Scientologist and producer of Lost and Cloverfield, announced a Cloverfield sequal.
  • In other Abrams news, Fringe (X-Files on drugs?) is being filmed in Toronto (UofT’s Bahen building, to be precise) as I write this. I dropped by today and saw an enormous FBI seal plastered on the lobby floor. Also, Great Hall has been revamped into an FBI HQ with suited agents running around. Very exciting.
  • Bonus fun item: I learned that Naveen Andrews (Sayid from Lost) has a fetish for older women. He fathered a child with his math teacher at the age of 16, and is currently dating Barbara Hershey who is a solid 21 years older.
No comments

Politics, photoshop, and Kanye

  1. The Black Eyed Peas’ will.i.am released an inspirational music video remixing Barack Obama’s “Yes We Can” speech.

    The remix features various people singing and reciting parts of the speech. The only non-English lyrics are Hebrew. Just saying. Nevermind, spotted some sign language, Spanish, and possibly some Asian dialect I can’t really hear over the music.

  2. There’s a collection of videos called “you suck at photoshop“. They’re both educational and amusing. Very well done.
  3. Did you know Kanye West has a blog? Yes, I know Britney Spears has a blog too, but Kanye actually posts interesting things.
1 comment

Code storytelling

It’s fun writing code that tells a story. Hopefully it’s also fun reading it.


class TerminationNotice(Exception):
    "Exception raised inside a thread when it's time for it to die."
    pass
 
class SuicideJob(QueryJob):
    "A worker receiving this job will commit suicide."
    def run(self):
        raise TerminationNotice()
 
class QueryWorker(Thread):
    "Devoted worker who will pull jobs from the `jobs` queue and perform them."
    def __init__(self, jobs):
        self.jobs = jobs
        Thread.__init__(self)
 
    def run(self):
        "Get jobs from the queue and perform them."
        while 1:
            job = self.jobs.get()
            if not isinstance(job, QueryJob):
                debug.error("%r ate a job that wasn't a edible: %r" % (self, job))
                continue
            try:
                job.run()
            except TerminationNotice:
                # Nice knowing you :(
                break
No comments

Installing Gentoo on Playstation 3

Mario!

  1. Back up my precious save files: Formatted my external hard drive to FAT32, plugged it into the PS3, went to System Settings > Backup Utility, and hit OK.

    While it’s backing up, I’m reading various Playstation 3 hacking forums, such as ps2dev.org where the first hack to utilize the locked-away GPU in Linux was created. Apparently there’s also hope to use the Playstation Eye as a webcam

  2. Burned the installcd ISO, inserted it into my PS3, hit Install other OS, it did its magic, rebooted, and I was in the Gentoo install CD. Just like that.
  3. Plugged in a keyboard into the PS3, created a password, started sshd, and ssh’d in from my desktop and did the rest from the comfort of my fancy chair.
  4. Followed the Gentoo Install guide which took about 20 minutes of work, the rest was waiting for things to download, extract, and compile. Meanwhile, I ate pizza and watched the first couple of episodes of Jericho through my PC. Not too bad.
  5. Time for the real purpose of this ordeal: Setting up emulators. After much trial and error, I learned that Mednafen is the cream of the crop. After more jumping through hoops, I created the perfect controller configuration that supports two PS3 Sixaxis controllers plugged in USB (bluetooth works too!). You can download it here: mednafen.cfg. Shove it in your ~/.mednafen/ and you’ll be good to go — full screen and all.

    To make your own key bindings, read the nitty gritty in this thread.

PS3 Mario

10 comments

Happy new year…

… to all my readers. Everyone else is not important.

<3
1 comment

Python humour

Inspired by xkcd.

3 comments

Found on the street

Found at King and Portland, Toronto.

1 comment

Linux Love, part 2

(Continuation of Why I Love Linux)

  1. Need to make a quick ISO image of your CD or DVD?
    dd if=/dev/cdrom of=/tmp/output.iso
  2. Your minion co-worker is leaving the office for an alleged 5 minutes, you believe David he is a liar and need to prove him wrong? time read in your nearest terminal, then hit ctrl+d whenever he’s back, and tada, instant stopwatch.
  3. Need to SSH but don’t feel like using ol’ cd and ls? Fire up KDE’s fish:// protocol and SSHing is the same as browsing your local files. Even edit things in-place.
  4. You’re doing something long and CPU-intensive in a terminal but you feel like playing a game that needs said CPU? ctrl+z to pause the process, do your thing, then fg to resume.
  5. Need to make a quick backup of something and too lazy to type it twice? cp foo{,.bak} is equivalent to cp foo foo.bak. Great for those /super/long_paths/that\ everyone.dreads.
  6. Bonus Super-Useful Alias: alias p='ps aux | grep -i'. Need to check if your rogue daemon process is still alive? p foo

Much of this was written about a year ago and I never got around to posting it. Yay for agelessness of Linux.

No comments

Next Page »