Lincoln's blog

[ Home | RSS 2.0 | ATOM 1.0 ]

Mon, 02 Jun 2014

Mon, 17 Mar 2014

Observations from Kenya

I'm in Nairobi, Kenya, sitting in our lawyer's office. He has a very strange glass table with a hole cut out of it. The hole is filled with another piece of glass which can rotate and acts as a raised lazy-susan. Weirdly, it's jointed so it can tip as well as rotate.

Paul is clearly quite a high powered lawyer. His office is on the 20th floor of a 20-story high rise in downtown Nairobi. The windows look out over the city and there aren't many buildings taller than this one.

He jokes with the other lawyers in the elevator: This is Bill, he runs my errands. (Laughs all around). No, actually, he's a partner. (Despite the joking, I pay Paul a tenth of what expensive corporate lawyers cost in the US, and he actually DOES run errands for me.)

The rest of the furniture in this office is very nice by Kenyan standards. It's about the level of a medium to low-end office in the States -- matched leather chairs but not particularly fancy, some of the leather is peeling, but nothing is ripped or torn. The low-shag carpet is kept very clean with no visible stains. There are laser printers and photocopiers and water coolers. In contrast, other places I've been to in Nairobi -- banks, etc -- are furnished more like a frat house with peeling upholstery, worn mismatched wooden chairs, and so on. No company in the States except maybe a very early stage startup would have furniture that ratty.

Oh yeah, and the power has gone out twice in two hours! Even Paul in this luxurious high-rise cannot afford to keep the lights on continuously. It's no big deal since the power always comes back within a few seconds.

Transit: Yesterday I took a matatu, today a city bus. They're both the same price (40 Ksh / 50 cents). In all cases, you get on the bus and then someone comes to collect the payment during the ride. There's always a conductor riding in the back whose job it is to collect money, open and close the door, and tell the driver when to stop. The matatu is essentially a 14-person van whereas the bus is an actual full-sized bus, with all seats facing forwards (as opposed to modern American metro buses which typically have many seats facing sideways, and more room to stand). The matatus leaving from the city center wait until they are full to leave, so they always squish everyone together. I haven't seen many instances of people hanging off the sides (though I was told to expect this). The matatus play deafening popular music from two giant speakers mounted on the wall behind the driver.

It's very hard to figure out where to get on the bus on the way out of the city. I've only tried once and I got on the wrong direction and had to take a taxi home. I even asked "46 Argwings Kodhek Road?" when getting onto the bus and the guy nodded and waved me on. But he apparently didn't understand what I was asking. Luckily, the ride took me somewhere special: Huruma, which is a slum east of Nairobi. I was hoping to see the slums but I didn't have plans to visit. The bus ride dropped me off on a dirt road where there were a lot of people living in makeshift dwellings. I was horrified by the stream running through the city - there were people sitting along it but it was just filled with trash. Essentially the stream was a landfill. It smelled awful. There was a backhoe moving trash around - I have no idea why. The landfill went on for like a mile before we turned off onto a bigger road.

Someone asked me if seeing the slums made me want to donate more to charity. The answer was no, not particularly. I already have a strong altruism drive, but it's not very sympathetically/empathetically motivated. It's more along the lines of "yes, I know people are suffering and we can probably do something about it. Let's make it happen." I did find myself slightly more motivated to succeed with my startup after coming back, but that could be caused by other things (like not having much time to get good work done for the whole trip, or reading motivational books on the plane).

Restaurants: The restaurant experience is pretty... odd. In all the restaurants I visited, the waiter brought me the menu and then just STOOD there waiting for me to order. My eyes flick to the drinks section and I order a drink quickly. "Anything to eat?" Oh, you want me to order my whole meal while you stand here? Okay then. I find the entrees section, scan it, give up and finally order whatever the waiter recommends.

I went to a Chinese restaurant. I put in my order for takeout and then sat at a table. They brought me a hot towel and some appetizers and I became confused because I wasn't sure if they understood what takeout was. The appetizers were actually really tasty: egg noodle chips doused in sugar, and a carrot & cucumber finger salad with a sweet sesame dressing. I scarfed it all down and then they brought me a carryout bag with my food. So I guess they understood.

I got fast food a few times. Pizza in one place, fried chicken in another. But fast food in Kenya is not particularly fast! It takes about 10 minutes.

In Nairobi there is a chain of "Java House" cafes. They are all over the place in the city and they cater heavily to foreigners/expats. To give you a sense: outside of Java Houses I saw maybe five white people the whole trip. Inside Java House, between the two times I went, I saw about twenty. They have recognizably American food like burgers and tuna sandwiches, as well as coffee, espresso machines, and DELICIOUS masala chai tea with milk.

M-pesa is a popular mobile money system in Kenya. It's what we're basing our business on - that nearly everyone has a way to receive money transfers onto their phone. It is very popular and definitely a household name, but it is not widely used as a payment method, unlike what I had expected. Most merchants do not accept it yet. (A local company, Kopo Kopo, is working on making "pay with M-pesa" a big thing.) But I did figure out how to use M-pesa to buy pizza from a pizza chain, and most taxi drivers also accept M-pesa.

Once I realized I could pay taxis with it, I started asking every time. It usually takes longer than just handing them cash, even if you have to wait for them to make change. But you don't get screwed when the driver doesn't have any change. One driver asked for a bit extra (+25 shillings on a 450 bill) to cover the M-pesa withdrawal, which costs a small percentage.

Getting a cell phone: I showed up at the Safaricom store at the mall the very first day I was in Nairobi. The store was packed and there were about 30 people in line waiting for help. Fortunately, there were only five people in line waiting for sales, but they all took nearly 10 minutes each, so it was about 40 minutes before I even got to talk to anyone about what I was trying to buy. What I wanted was slightly complicated -- a new phone, a GSM modem, two SIM cards and an M-pesa account for each SIM card. I got as far as talking to the person and telling him all this when he handed me a big form and said "fill this out". Of course, since the store is packed, there's nowhere to fill it out, so I ended up using one hand to support my pen writing through the paper.

They are out of the cheapest phone so they sell me the next one up. It costs 6000 shillings, about $70. I bought with my US VISA card. I only bought 100 shillings ($1) worth of airtime for it, which lasts about 10 minutes of talk time. I ended up buying another $5 worth of airtime during the whole trip - so at least that's relatively cheap. The GSM modem was about $20 and the SIM cards were free. To get the SIM cards I had to show them my passport. Then I had to go to the M-pesa desk and register my SIM cards with M-pesa, which required showing my passport again. Fortunately, nobody gave me any trouble about being a foreigner.

Being white: There seem to be a few people in the city who prowl around, looking for touristy-looking white folks. Two days in a row, the same guy found me on the street and would chat me up, teach me Swahili words and act really friendly. It was obvious, though, that he was trying to bring me into stores (perhaps on commission) or taxis. I actually used him to find me a place to buy a plug adapter I needed, and having friendly human contact (however insincere) was still kind of pleasant.

But the real interesting one was that while I was waiting for the fried chicken, I was accosted (or maybe awkwardly hit on?) by a local, a girl named Carol. She was very polite. She sat next to me and started asking me questions. She was a student studying biotechnology at the University of Nairobi. She wanted to move to the US since some of her family lived there, but they would not give her a visa since they thought she would stay as an illegal immigrant. She added, "and they're probably right!" She wanted me to look at her visa application and try to see if I could make any suggestions, but I declined to help in this case. (And in case you're wondering, no, she was not particularly attractive.)

The few instances when I would pass white people on the street, or ride elevators with them -- quite rare, and I always felt a bit of companionship. I guess this is how people who look different feel most of the time in the US? Definitely good to have experiences like that.

Everyone was dressed like a businessperson in the city. There were very few t-shirts -- basically everyone wore at least a button-up shirt, and lots of jackets and slacks. Long sleeves were quite common but lots of short sleeves as well. The weather was warm enough (high 70s F) that in the US I would never wear long sleeves. When I would talk with taxi drivers they would always comment on it being "cold" which I found funny.

Sunscreen costs $15 a bottle (I guess not many people need it?). But deodorant is very cheap! And every taxi driver made some comment about Obama when I said I was from the US.

The security! Oh! Let me make a quick list: Checking the trunks of cars as they enter any private parking lot (malls, office buildings, etc.) Showing the contents of my bag at the entrance to malls, office buildings, and restaurants. I probably got wanded about 8 times a day, and had to open my bag maybe 4 times per day. There's a guarded gate to the entrance of every apartment complex. My taxi got stopped at night in a police check on the street - they were stopping everyone and shining a flashlight into the car. My hosts told me I should not walk around after dark or ride matatus - since I'm white I am an attractive target to be mugged. Of course, I did ride matatus and wander around after dark, and never saw anything resembling danger, but maybe I just didn't get unlucky enough (or maybe the danger is overblown). Walking around after dark was quite scary, but mainly because there are no streetlights, few sidewalks, and the cars go fast on narrow streets.

Modern conveniences: "The Junction Mall" is a very much a US styled shopping mall. It's in the western Nairobi suburbs, not too far from where I was staying, in an area where richer people live. The only difference from US malls is that there are no stores with entrances to both the mall and the outdoors, so you have to go in through the main mall entrance. But this was a full size mall - multi story, food court, fashion stores, cell phone stores, video game store, even a Subway. Perfectly clean and well maintained. Free wifi in the food court. Actually, there was a surprising availability of wifi in the central business district of Nairobi as well -- not too hard to find, and for relatively cheap (under $1 for 24 hours of wifi).

But the real treasure at Junction was a two-story store at one end of the mall. This was unlike any store I have seen in the US. The closest is probably Carrefour in France. This store sold (deep breath): all kinds of groceries including health food, basic clothing, fashion clothing (including sub-stores like a Skechers shoe store), books, kitchen supplies, home goods, refrigerators, indoor furniture like couches, outdoor furniture, camping supplies, gardening supplies, yardwork tools, lawn mowers, motorcycles and home gym equipment. Also, there was a woman at a kiosk who offered to make you instant ramen on your way out of the store.

I think that about sums up Nairobi.


posted at: 03:48 | path: /projects | permanent link to this entry

Mon, 06 Jan 2014

Problems I Solved Making My 3D Printer

I got a Velleman K8200 3D printer kit for Christmas, and I just got it working today.

It took me about 28 hours of assembly to get working. 20 of those were following the online manual and the last 8 hours were fixing problems that cropped up.

Here's a Youtube video I recorded of it printing, and me pointing out a few things. Sorry that the quality of this video isn't high.

A few photos of the first thing I printed. 1 2 This casing was a large print - it took 4 hours to print. It was this printer's recommended "first print"; it's a box to house the controller board on the printer itself.

Anyway, I thought you might be interested in the problems I encountered after completing the assembly.

First, the Y axis motor wasn't moving the bed correctly. This took me a long time to diagnose. At first I thought it was a problem with the belt or the bearings, since the bed was a little hard to slide back and forth. But I lubricated it and the bed got a bit easier to slide, but the motor still had trouble pushing the bed. Then the belt fell off! I had apparently not attached it properly in the first place. So I fixed that, but it still wasn't working.

So I decided to dig in. I cut off the heatshrink and found a disconnected wire. Apparently I had failed to solder it properly, whoops! I resoldered and re-heat-shrunk all the connections. And it still wasn't working properly. Argh!

I removed the belt and put my finger on the motor gear. The motor itself was turning, but my finger could easily stop the gear... Oh. Duh. I tightened the screw which affixed the gear to the motor shaft, and it started working! Hooray!

The second problem was that the extruder would get "stuck". The extruder is the part that pushes the filament through the hot end -- it deposits the plastic. The extruder would work for a while and be happily pushing out plastic, and then it would sometimes get stuck and not be able to push the filament. If I used my fingers and turned the gear manually, it would often "unstick" and start working again. The extruder has a fancy adjustment bolt which lets you tighten and loosen its grip on the filament, so at first I thought this was the ticket and I spent a long time tweaking that bolt. But no dice.

I took apart the extruder. When I disconnected the stepper motor I noticed that even when there was no filament, the extruder wouldn't turn smoothly -- it would have "easy parts" and "hard parts" of its rotation. So the fix here was to loosen the main rotating bolt (the "hobbed bolt") that the gear was attached to. When I did that, the extruder turned a lot more smoothly, and on reassembly it had no problem pushing the plastic through.

But now when I printed, the plastic wasn't sticking properly to the bed. I adjusted the height of the bed endlessly, thinking it was a height problem, but it wasn't -- the bed was just not the proper material. I taped over it with blue painter's tape, as many people recommend on forums, and finally -- FINALLY -- I could print. Hooray!

posted at: 02:26 | path: /projects | permanent link to this entry

Sun, 28 Apr 2013

Meditation practice

I've gotten back into doing regular meditation. The last time I did it regularly was at rationality camp. The nudge here was reading this page on mindfulness engineering, as well as starting to use Beeminder -- I wanted a simple habit to test Beeminder with, and I decided on meditation. (My Beeminder binds me to doing 100 minutes of meditation a week.)

The method I use is to sit in half lotus, on a cushion, for 20 minutes, and concentrate on the sensation of breath entering and leaving my nostrils. My brain doesn't like focusing on this for more than a few seconds at a time, usually. At the beginning of my meditation sessions I get distracted by plans, fantasies, memories, and so on. But by the end of the sessions I get distracted by pain -- my legs tend to fall asleep. I've only been doing this for a couple weeks so hopefully the legs falling asleep will get better.

At least two of my friends independently decided to also start meditation practice recently. I didn't find out why they started yet, but it should be interesting to see which of us continue and which quit, as well as comparing what effects we observe.

Observations: When I drink caffeine I seem to be slightly better at concentrating on the breath. When I drink alcohol I seem to be worse.

Goals: Someone asked me what my goals were for meditation. I told her five things: build a habit; practice attention control; take refreshing breaks from work; practice sitting quietly; and obtain better control over own emotional states.

But if you are interested in it, I recommend the free book Mindfulness in Plain English (MPE). MPE claims some interesting sounding benefits, such as "recognize desires but not be controlled by them"; "your arrogance evaporated and your antagonism dries up and your life smoothes out"; "life becomes a glide instead of a struggle"; "sharpens your concentration and your thinking power"; "you come to a direct knowledge of things as they really are".

As for me, in the couple weeks I've been doing it, I feel a little more poised in daily life. When I sit to work after meditating, I feel a little more precise in what I work on. This is giving me a small amount of positive feedback, so I expect to continue doing it. I don't feel like I've gotten any better at attention control during the meditation sessions. This is what I expect to improve first.

posted at: 05:18 | path: /projects | permanent link to this entry

Sun, 18 Apr 2010

Introducing Break Ideas

I mentioned in a blog post that I had this idea for a project called Break Ideas. Well, it's working now, sorta. I started working on it because at BarCamp Boston this weekend, there was a programming contest where you were supposed to learn a new language and implement something useful in it. Well, I learned Arc for this project.

GitHub project page

What is Breakideas? It's an idea sharing site. I started to spend a few minutes just letting my mind come up with creative thoughts each day, and I felt like I was discovering a lot of interesting and occasionally useful insights about technological and social issues. Eight minutes is the length of my twice-a-day wrist break, and it seemed like a reasonable length of time to think -- not so short that I couldn't develop any ideas, but not so long that it takes a significant chunk out of my day.

How should I use it? Go to the site! Take a break! Think! Write a summary! If you can't think of anything, other people's ideas are provided to give you inspiration. Pick one of them and develop it further. If you want people to be able to contact you, put your contact information in your profile.

Thoughts on Arc after about 6 hours using it:

I feel like I can see into PG's brain!

Paul Graham, PG for short, is the designer of Arc. He wrote Hacker News in it. As far as I know, News is the only substantial production app written in Arc.

The language is basically a dialect of Scheme, but it has a very different feel from the little bit of Scheme that I've done. There are a TON of little macros and shortcut functions that capture a pattern that showed up -- one I liked was iflet which, given a variable and a test (as well as a then-body and else-body), binds the variable to the result of the test, then executes the then-body or else-body depending on the truth of the test. It's a useful idiom for when something can be nil. The sheer number of these little shortcuts makes reading other people's (pg's) code hard, because while I can sometimes guess at their behavior by name, the precise semantics can be hard to divine.

Other than the pile of macros, the language is very satisfying to program in. I was successfully able to read, understand and modify the blog.arc program (distributed with the arc source) to do what I wanted, and in a fairly short period of time I had something functional.

I liked Arc for its simplicity in implementation -- looking at the source code and making tweaks was extremely feasible, as well as the many features it provided to reduce friction in web development.

I had trouble with the heavy usage of macros in the example code (treating them like functions is a bad idea), as well as the built-in user authentication system (it seems quite inflexible). The development environment REPL currently requires 3 commands after pressing Ctrl-C in order to restart the server and pick up your changes (one in MzScheme to get back to Arc, one to reload the code and one to start the server).

Overall I'd recommend it to anyone who wants to do hobby web development, and doesn't mind working in an environment which always feels like a prototype -- good because it's simple enough to be easy to change, but bad because it doesn't seem robust.

posted at: 08:55 | path: /projects | permanent link to this entry

Sat, 13 Mar 2010

JIT review paper

I am compiling links and notes to papers about JIT bytecode techniques. Eventually, this collection will become a full-fledged review paper.

Tracing JIT compilers

Good discussion and links here. Highlights: LuaJit, Dynamo

Mozilla JagerMonkey

Interesting hybrid approach.

Equality Saturation

Not extremely JIT related, but interesting nonetheless. Instead of applying peephole optimizations, a model is constructed which represents all programs which do the same thing as the input. Then the problem is treated as an actual comb.opt problem and the answer is searched/solved for.


Dynamo is a JIT translator from a native code to itself, optimizing in the process. Most gains are probably from turning complicated control flow into straight-line code. Transmeta Crusoe was a hardware-accelerated version of a similar concept.

posted at: 05:40 | path: /projects | permanent link to this entry

Tue, 16 Feb 2010

Solving the procrastination problem

I love sites like reddit and Hacker News. They occupy me when I'm bored at work, and they expose me to a huge number and variety of comments, blog posts and news stories, so I feel like I am learning a lot about a lot of topics.
They give me the sense that I'm staying current on technology.

However, over the last two or three years, Reddit and HN have become the "go-to" sites when I have a browser open. I need only to type a single letter in my browser in order to visit them. Nearly every time at work that I tell my code to compile, I tab over to Firefox and refresh HN, looking for something new, any snippet of information that will capture my interest for a few short minutes. And it's all nearly unconscious at this point. Recently I idly typed in the address bar when I was already at I noticed, and was surprised, only because the page didn't change -- I didn't get my fix.

And the addiction metaphor is apt. I have observed my own nearly unconscious, highly trained behavior to go to a new page, scan down the page for unread headlines, and learn a new tidbit of advice from a blog about Haskell, JSON or entrepreneurship. As I understand it, learning is connected with dopamine, so it sounds reasonable that someone could become addicted to it -- dopamine is implicated in lots of other addictions, including gambling, drugs and sex. I also noticed that my "learning addiction" is interfering with other aspects of my life, specifically my own sense of productivity; there are plenty of days where I come home thinking "if only I hadn't spent so much time on Hacker News, I might have gotten all this other stuff done". And I beat myself up over it, a little bit.

I can't be alone. HN has a "noprocrast" feature, where it will give you a friendly message and prevent you from visiting the site if you come too often. The existence of this feature makes me think that other people have some form of this addiction too. But the feature is frustrating to use: every time I am denied my fix, I get a little bit annoyed, and when I'm annoyed, I don't go back to my work -- instead I visit Reddit or Slashdot to cool down and get my fix there. It's much harder for me to focus when I'm frustrated, so it sort of makes the problem worse. It doesn't help that HN's noprocrast message shows a timer (you can't come back for 30 minutes!) which lets me look forward to exactly when I can come back, and that's distracting too.

So I'm looking for other ways to train myself off of this habit. So far, while writing this post, I already had the urge to visit HN four times. Three of those times, I noticed what I was doing, and stopped myself from loading the page. Once, I did actually load HN; I was thinking to myself "I shouldn't be reading this, I have a blog post to write, but I just want to see what's new on this one comment thread."

One way of easing my addiction could be reducing the delivery rate of the drug. If HN had more momentum (for example, if the top stories changed more slowly), it might reduce my desire to come to the page, because my expectations of getting my fix would be lower. Then again, maybe it would just cause me to look at the "new" page more often, which just shows the 30 newest submissions, regardless of quality.

Here's a more radical idea: what if the whole site only updated once every four hours? No realtime comments, no stories, etc.; when people make submissions and comments, the site queues them up for the next big update. This would still have the countdown-timer effect, giving me something to look forward to, but there would be no incentive for me to refresh the page between those moments. (I note that you could implement something like this cheaply using an HTTP proxy by tweaking the caching headers.)

I am also trying to come up with productive activities which compete with browsing for when I'm bored. I don't always want to think about work during compiles, but I also don't want to visit Hacker News and lose half an hour or more.

So what makes me feel productive that I could do at work? Well, when I get a few minutes to brainstorm, I sometimes come up with ideas which seem original and worth keeping. It seems like browsing Hacker News directly cuts into time that I could otherwise spend just thinking, away from the computer and other distractions. So here's my idea: when you're bored, you go to this site, and it presents you with some other people's postings as inspiration, and instructs you to spend eight minutes brainstorming about whatever you want. You step away from the computer, come back in eight minutes, and the site asks you to write down what you thought about. Then the next person who comes to the site might see your idea as inspiration.

Clearly there are some issues -- it would be nice to have usernames and attribution -- but I think the idea is sound, and I know that I would probably contribute to it sometimes instead of visiting HN. I know from experience that it takes some mental effort to choose to produce something new instead of just consuming, but I am willing to put in that effort if it makes me feel less like my time is being wasted.

posted at: 01:50 | path: /projects | permanent link to this entry

Wed, 03 Feb 2010

My blog is working again

Blog was broken because of a server upgrade. It was broken because the code I'm using (pyblosxom 1.4) was testing for the presence of a feature and behaving differently.

The feature is the standard library's wsgiref. If the wsgiref module is present, it tries to use it instead of just calling the CGI script directly. Unfortunately the wsgiref module uses some settings for PYTHONPATH that I didn't bother to try to figure out, so the script broke.

posted at: 04:09 | path: /projects | permanent link to this entry

Mon, 13 Apr 2009

Cavendish's Physics Experiment

Joe Marshall posted the details of his replication of the Cavendish experiment on his blog. At the bottom he didn't do the calculations, so I got excited and worked them out.

Here's my math:


F = ma (Newton's motion)
x = sin t/(2pi*T) (simple harmonic motion with period T)
F_s = -kx (Hooke's law)
F_g = G m1 m2 / rad^2 (newton's grav.)


m1 = 0.9kg
m2 = 3.6kg
T (period of oscillation) = 40 min

To find the distance between centers of mass, for the G calculation, I had to guess based on the objects he used.

He said 1 inch, I assumed this was one inch separation between edge of the water jug and edge of the lead cylinder. Assumed radius of milk jug = 9cm; lead cylinder = 2.5 cm; separation = 2.5cm, for a total of 14cm between center of mass.

rad_g = 0.14m

I also had to assume he placed the centers of the lead cylinders at the end of his 3ft dowel.

rad_dowel = 1.5ft

And I also assumed that he was projecting the laser onto a perpendicular wall (20ft away):

rad_wall = 20ft

light moved "1 or 2 inches": I assumed 3cm

delta_x_wall = 0.03m


First, find the spring constant, k, for the hanging system.

Doubly differentiating the harmonic motion equation and solving leads to:

a = -x / (2pi * T)^2

Substitute f = kx and f = ma

k = m / (2pi * T)^2

Use m = 2m1 (2 lead weights in motion). Neglect the mass of the dowel. I got

k = 1.9 * 10^-5 kg/sec^2

Now we can figure out G. By similarity of triangles,

delta_x = delta_x_wall * rad_dowel / rad_wall

By hooke's law, F_spring = k (delta_x). Since we are concerned with the system's new rest position, total forces are zero so F_gravity = F_spring.

There are two instances of gravitational interactions we're measuring which sum:

F_spring = F_gravity
k*(delta_x) = 2 * G * m1 * m2 / rad_g^2

Plug in values and solve for G.

G = 1.32 * 10^-10

whereas Google says

G = 6.67300 * 10^-11 m^3 kg^-1 s^-2

(too big by about a factor of 2)

posted at: 01:00 | path: /projects | permanent link to this entry

Fri, 18 Apr 2008

A Fitting Analogy

I thought of an analogy for our project that seems to fit fairly well. We're the "relational database for code". Relational databases change the way people think about storage. Before they existed, if you wanted to store some data, you opened a file and put whatever you wanted into it. When you read the data, you have to parse the file format, and you can only read the file sequentially, and so on and so forth. There are tons of different file formats that people use, but everybody does it differently, and if you wanted to be able to use the data in a lot of different ways, you had to think very hard about the different ways and design a file format that allows you to be efficient about it -- leading to greater complexity, bugs, and poor flexibility.

Enter the Relational Database. In order to use a relational database, you have to think a little about the data you intend to store. You have to organize it into relations: what pieces of data are associated with what other pieces of data, etc. You also have to tell the system a few other details about how the data will be used -- indexes, views, etc. But once you do this, it's massively easier to deal with all sorts of data. It's more flexible, because you can write queries that ask the database all sorts of questions that you couldn't have anticipated while you were creating it. Relational databases are also better performing, because the database creators were able to put in a lot of advanced technology to use indexes and perform query optimization and so on that you would rarely have the resources to be able to write yourself. Both this performance and this flexibility are gained from telling the database some critical information about the data that you're storing; you can't generalize the idea of a join between relations without the user telling you what is in a column.

We are aiming for a similar paradigm shift for software, especially communicating software: tell the system a little bit about how your functionality is structured -- what operations apply to which pieces of data. Stuff that you probably know anyway, or can figure out. Once you've explained this to the system, the system provides lots of flexibility (through making it easy to send messages, implement new objects, etc.) and performance (through parallelism, optimization, distribution, etc.).

posted at: 01:40 | path: /projects/x | permanent link to this entry

Sat, 05 Apr 2008

Structuring your app with Units

A major component of our design for a message-passing framework is the idea of a Unit. A unit in our language is an aggregation of data and code that can receive and send messages. It's analogous to an object in object-oriented languages.

Note that I said analogous. They're defined similarly to objects in that you specify fields and functionality, but you have to treat units differently from objects because of their runtime characteristics. Specifically, at most one message that uses the state of a particular unit can execute at once. This has extremely interesting implications.

For one thing, units' state is consistent between messages. What do I mean by that? Well, contrast it with most shared memory systems, where if two threads are trying to increment a shared counter, one thread might interleave one read operation between the other thread's read, add-one, and write sequence. If this interleaving occurs, then the system is brought to an "inconsistent" state. We obviate synchronization problems like this one -- the analogous problem is that an incrementer unit receives two simultaneous messages to "increment yourself", which is defined as read, add-one, and write. However, the runtime ensures that the two "simultaneous" messages are actually executed in sequence, so the problem above never occurs.

When I say that only one message on a unit can execute at once, I actually mean something a bit different: only one message appears to execute. In fact, the runtime might be executing many messages, but they will never notice. Another way to put it is that the runtime enforces that messages are transactional upon the state of the unit: messages appear (to observers -- other messages) to happen all at once and in isolation from other messages.

These transactional semantics have great implications to the design of systems. Traditional object oriented programming requires the programmer to break up their code by functionality. While designing a Web server, for example, a reasonable object-oriented design would include a module that listens on the socket and provides events, one that parses the HTTP protocol, one that looks an object up in the VFS based on the GET string, and one that spews the object back to the user in the HTTP protocol across the socket. Note how this is segmented by functionality -- take a moment to think about what features of the code are required at each level. Now think about how these modules communicate. The event comes in. That code (the event handler) probably calls a parseHTTP method to achieve an HTTPMessage structure. Then the event handler passes that to the parseGET method, which calls the vfsLookup at each level of the GET string until it reaches a terminal object. It returns the object. Lastly, the event handler passes the GET string and any query options, post data, and the socket descriptor to the object and tells it to handle the request and send the user the data. For file objects, the object just sends the mime type and the data. For other objects, some special code might be run.

In our system, you can organize your functionality in this way but it's more important to think about the data. What data is involved in handling a web request? Clearly each web request is a separate event to the server, and they have no shared state. So the incoming socket listener is probably bound to a "free operation" -- a message with no unit attached. This free op accepts data on the socket, parses it (using an HTTP parser library), parses the GET string into components (using another library), and sends these components to a VFS unit which knows how to do the lookups. This unit may just be a front-end to a tree of separate VFSDirectory units. Anyway, once the path components are resolved into a target object (itself another unit), the object is passed a message with the correct arguments as well as the capability of writing to the socket (derived from the event) and is expected to write its output to the socket in the same way.

So what's different? Units make the user focus at first on the data, not on the code. Only after the system is correctly separated into units do we think about organization of functionality, which is nearly orthogonal to the question of how data is broken into units.

Code and more case studies to follow.

posted at: 09:11 | path: /projects/x | permanent link to this entry

Tue, 27 Nov 2007

Computation in X

If one were to examine computation in X, what properties hold?

If we ignore objects in the language, computations are deterministic. This will be demonstrated by breaking computations down into a series of steps and examining the results of each step. If each step is deterministic then the whole computation is deterministic. But what about steps being executed in parallel by an adversary scheduler? I'll show that even if you don't know the whole state of the machine at once, you can always predict the pieces of state that could possibly be needed by the current step.

The absence of "mutable shared state" will be key to the proof. In X, we promise to the user that there is no such thing as shared state between concurrently executing threads, except in the context of objects and their member variables. We will prove this by examining each syntactic form in the language and showing that it cannot create a reference shared with any component that could execute at an unknown time (again, in the absence of objects). If this property holds, then we guarantee that at each step, all references available to the step are either to immutable objects or to mutable unshared objects.

Now, if the reference is to an immutable object, then it must be in a known state: the state it was created in. If the reference is to a mutable unshared object, then it must also be in a known state: the state obtained by starting with the state it was created in, then applying all operations to it in order up to the current step. Since the object is unshared, there is no point where the order of operations is ambiguous. Thus all references are in a known state at each point and thus the computation is deterministic.

We would also like to show that each task, once started, is wait-free: it completes in a finite number of steps (it never has to wait for another task). Tasks may be spawned by other tasks (via send), optionally passing futures. The task is never started until all the futures in its arguments are filled.

This proof is simple, as there are no waiting constructs in our language. In order to "wait" for the value of a future, you have to make a new task which receives the value.

posted at: 23:40 | path: /projects/x | permanent link to this entry

Thu, 15 Nov 2007

My resume

I recently updated my resume. For the past year or so, I've had my resume in an abstract XML source format I came up with. (I'll put up the code when I have time.) I then use XSLT (Extensible Stylesheet Language Transformations) to convert from my XML source into XHTML, which you see on the resume page.

XSLT is a declarative language. You specify what elements you want transformed and what the output is, and it goes through the document and applies your transformations where they match. It's kind of enjoyable to code because there's no tedious "loop through the DOM" sorts of affairs like you might write if you were making a Python XML processor. (Although the XML syntax is its own kind of tedium, so it's a tradeoff.)

I then wrote another XSL stylesheet to transform my generated resume into the Web "clean" version linked above, which removes my email and mailing address. (I only want job offers from real, intelligent people, and my Turing test is devising a way to contact me.) It also replaces the CSS to accommodate the Web format a little bit better. (The 'print' version is almost the same, but the font sizes are a little different.)

My resume is built using a Makefile on the Techhouse server, which does almost all the steps: xsltproc (the XSLT stylesheet transformer tool) from the source to the full version, xsltproc from the full to the clean version, and xmllint --format to pretty-print the XML. But The one thing I haven't gotten a good automated procedure for is converting my resume into PDF. I used to like the result of printing it to PDF using Firefox on OS X. But I won't always have OS X available, and I don't know any way to automate that. So it's not the best situation. I tried a few 'html2pdf' type tools, but they produce poor quality results.

posted at: 18:34 | path: /projects | permanent link to this entry

Tue, 06 Nov 2007

X the language

Sean, Tara and I are working on a concurrent programming language. It's based on message passing. Message passing, you say sarcastically, that's a new idea: look at Smalltalk and its object oriented progeny like Java.

But our messages are different! They're asynchronous. Instead of sending a message and waiting for the response, you continue on your merry way.

You may have had this idea before. I did -- it's not really a difficult concept. The question is, when you need to get a value back from that message that you sent (like a return value), how do you do it? Stop and think about this. How do you do it?

One idea is to send a return port along with the message, and have the client send you data along that port. This is okay, and our language certainly supports it. But what if the port needs a bunch of different pieces of data as results from a bunch of different messages? You have a serious data collection problem, where all your return ports are invoked at different, unpredictable times.

Okay, so instead, let's try this -- when you send, you get back a token (let's call it a Future) which you can "force" in order to retrieve the value. When you force a future, your message blocks until the desired message produces a value, and then you get it. But let's add another constraint: messages should never block. (I'll get to why this is useful in another post.)

So what, then? Here's our proposal: sending Futures as arguments to a message causes them to be resolved before that message is delivered. That's kinda weird... but it solves our blocking problem -- the only "blocking" that occurs is that a message's delivery might be delayed, but since they are asynchronous, it doesn't matter. It also makes it easy to do the data-collection problem, as you can start a bunch of asynchronous messages and then pass all their futures to the same place -- the place where you do your work depending on those values.

posted at: 09:43 | path: /projects/x | permanent link to this entry

Sun, 13 Aug 2006

Taking Apart My Powerbook

So the other day I dropped my Powerbook. I was on the train and I had it on my lap, shut, because I had been working and decided that I was getting tired. So I dozed off and I'm thinking "well, if it starts to fall I'll just wake up and catch it." Well, as I'm sure you can guess, I didn't wake up in time. It hit the floor and made a pretty loud sound. Oops.

I immediately noticed that my sleep light wasn't working. I was sad, but I figured I could live without it. The next day I noticed that not only was the sleep light broken, but so was the sound! Uh-oh. All of a sudden this is a much more serious problem. I wonder if I can open the thing up and possibly tweak something to get them working again, or at least see what's broken. So I go start googling guides to take powerbooks apart, because they're constructed in a way that makes it a bad idea to do it without a guide. Well, I found one: iFixit. It has specific instructions for everything, so I clicked on "sound card" and read through the whole thing. As it turns out, it claimed, the sound card and the sleep light are on one particular connector on the bottom left corner of the machine. Aha, I say, maybe that connector's loose. So I go to the hardware store and pick up the appropriate sized screwdrivers (#0 phillips and #8 torx) and go to town. Man, taking apart such a delicate contraption is a blast.

And sure enough, when I get the thing open, the connector's loose. I pop it back on. While I have it open, I'm poking around. My latch has been broken for years -- I've always had to give the lid a little "tug" when closing in order to get it to stay closed. Well, now that I have it open, I see a tiny screw stuck to the magnet of the cover. I extract the screw and save it. I test the latch a few times and it works much better, and this makes me very happy. Anyway, when I was opening the computer, I had to only take out one of the three expected screws from underneath the battery. But I found one stray appropriately-shaped screw rattling around inside the case and another stuck to the magnet. They fit into the battery place. So this Powerbook opening was really an awesome project, because I actually made my machine work BETTER afterwards.

posted at: 08:46 | path: /projects | permanent link to this entry

Wed, 18 Jan 2006

How to build a kotatsu

My roommate Haynes and I built a kotatsu in our room. We built a simple 3'x4' wooden frame out of pine two-by-fours and four-by-fours, and cut a tabletop out of half-inch plywood. We put a king-size comforter over the frame, and the tabletop over the comforter. We finished the tabletop with a router, then sanded, stained it (red mahogany stain), applied tung oil and finally sprayed lacquer. It ended up shiny and quite waterproof. Pictures will be added soon!

I wish we had a better story around it: one night, at 4am, Haynes said "we should build a kotatsu" and told me what it was. I said okay, let's do it, so we both chipped in 20 dollars to buy a heater online, before we went to bed. The next day, we both had it on our minds -- and that evening, we went downstairs to the workroom and built it. It was a relatively easy project, but it was still very satisfying to see it all come together. Three hours and we had the frame done. Over the next few days, we purchased a comforter ($65 or so from Bed Bath and Beyond -- they had a special deal), finished the tabletop with cheap hardware store materials, and began using it in our room. The only thing that hadn't arrived when the semester ended was the heater itself. It should be there when I return, though!

It has really become a great social center of the First Floor of Techhouse. People come and hang out under the thing, play video games, or whatever. It's very awesome.

posted at: 04:33 | path: /projects | permanent link to this entry

Mon, 02 May 2005

How to take apart an iSight

First unscrew the screws on the top and bottom of the Firewire port. You'll also want to pop off the little white protector ring around the port -- there are four plastic hooks, on the top, bottom, left, and right. Insert a flat screwdriver to pop them off.

Now slide the metal perforated casing off the back by gripping the camera and pushing on the Apple logo. When you get to the end, it might take some extra pressure. My Apple logo popped itself off at this point, which was fine; if you have trouble getting the casing to slide off, you may want to pop off the logo beforehand.

Okay, you're done with the mystery part. You can now unscrew the three screws around the lens. The lens slides off the front, you'll have to do some wiggling, but don't push it too hard, because you're still attached via cables. This is where I stopped -- I could see some screws on the lens piece, but I didn't want to detach the cables in order to unscrew them, so I put the whole thing back together at this point. And it still worked! Hooray.

posted at: 07:06 | path: /projects | permanent link to this entry

Wed, 27 Apr 2005

Project Manager!

I am quickly rising through the ranks of TH :) -- I've been elected Project Manager and Haynes, and I have also been appointed to join the ranks of the server roots. The root password was decided today. No bribes are currently being accepted, but watch this space!

My election as Haynes was probably the most interesting. A couple months back was the semester officer review, where everyone pretty much got to praise the officers and take a vote of confidence, as well as giving them feedback. Well, Haynes is pretty high profile and people like making fun of him, so we had a vote of confidence for Haynes, and he received a resounding no (7-4, I think). So he's due to be replaced as Haynes. I and three other people (Haynes himself, Tara, and one other person whom I forget) ran for Haynes at the officer elections, and I won, 18 (me) - 17 (Tara) - 7 (Haynes). With a clear mandate among the 25 or so voting members, I am happy to accept the position of Haynes for the upcoming year.

Projects? Awesome. Elite, even. I love projects and I love Techhouse and it's gonna be sweet :). I'm sure you'll hear about them incessantly in the next year or so!

posted at: 06:00 | path: /projects | permanent link to this entry