This, he says, is Satoshi, and I applaud Sergio for this clever way to recognize an individual miner. Like Sergio, I am pleased that Satoshi’s fortune in Bitcoins is now apparently worth around $100 million USD. But Sergio also suggests that he expects this will lead to the unmasking of Satoshi once others track this to a Bitcoin somewhere which HAS been spent. (Bitcoin has many advantages, but it is NOT fully anonymous: in fact,  anyone can track a payment back to see which (anonymous) account it came from previously.)

I hope he is wrong about the unmasking. I prefer to imagine that Satoshi Nakamoto is living and working a normal job, still haunting cryptography boards in the evenings and on weekends, and occasionally checking the news to see how that Bitcoin thing is progressing. I imagine that someday, many years from now, when she dies her husband will open that envelope she left in the safe-deposit-box and it will contain a hard drive and stack of papers labeled “Now that I am gone, please publish this for the world to read.”

Okay, it’s just a romantic dream, but I’m hanging onto it as long as I can.

But Mr. Kumar didn’t write me about a position commiserate with my specific skills, he wrote to tell me “We have more than 100 W2 working currently with successful hit.” (That’s not quite English, but it’s fairly close.) There are recruiters who work hard to match up a particular applicant with a position where their skills and their career/environment preferences are a good fit. When I am doing the hiring (and just to note, Capital One is hiring right now in the Wilmington area), I love working with these recruiters: they bring me just 3 resumes and I end up wanting to bring in all 3 for further interviews. That’s a much more pleasant experience than digging through a stack of resumes most of whom can’t pass the FizzBuzz test.

Mr. Kumar is in a different category altogether: he clearly thinks recruiting is a numbers game: if he just sends enough applicant names to enough open positions then he’ll be successful. He won’t be, because he’s not adding value. So I politely wrote back to Mr. Kumar explaining this and asking that he not send me “blind mailing” style job offers. A week later I have received TWO other emails from Mr. Kumar stating that “Panzer Solutions is looking to hire 10-20 New H1b’s and OPT EAD’s in coming one month.” (Still, not quite English.) Besides being a violation of federal employment law (I’m not a lawyer, but I was under the impression that companies were not permitted to favor H1B holders over citizens), this is no better than spam, either for the recipient (me) or the employer to whom the names are offered.

So I am Naming and Shaming Mr. Sunil Kumar of Panzer Solutions, and I will never do business with him or his company. Here’s hoping this article jumps to the top of the search rankings for those names so that others will recognize their uselessness sooner and Panzer and Mr. Kumar can quickly go out of business and leave space for better recruiters who actually make the hiring process easier, not harder.

• Talk to it. Today I *almost* have this: my Jellybean-based phone can perform near real-time voice recognition (without a network connection). But the error rate is still high enough that after adding in the time to go back and correct errors the whole process takes longer than typing it in on the device’s keyboard. But not much longer… I expect this one very soon.

• Context aware. My phone should know when it’s OK to ring, and when it isn’t (if I’m in a meeting or a movie). While I’m driving, it shouldn’t send me texts. When I start asking for directions it should guess (with a degree of accuracy) where I might want to go to.

• Expandable screen. I want an iPad sized screen, but I want it to fit in my pocket. The only way to do that is to have an expandable or pull-out screen of some sort, or perhaps a projector.

• Keyboard. Something real that I can type on — keyboards are SO amazingly effective. But I don’t like carrying around a bluetooth keyboard (they’re either too small to type on or too big to carry comfortably).

I believe that story points serve as a “rough” estimate. In the teams I work with, story point estimates are made quickly (a few minutes to be sure we understand the story, then quickly discuss and reach a consensus estimate). They are quantized (must round off to some Fibonacci number) which means that any given estimate is necessarily imperfect.

As such, they provide a cheap (didn’t take long to generate) but rough (not perfectly accurate) estimate, and they have to be respected as such. Story point estimates would not be useful to answer questions like “Will this project deliver in October or November?”, but they ARE useful for questions like “Would this be a 3-month project or a 1 year project?” For some purposes, a more precise estimate is needed, and then it may be necessary to invest a few hours to a few weeks to perform detailed work to generate a more precise estimate. However, I think that such situations are rare: people *want* perfect estimates ahead of time but rarely *need* them. Also I think that people are usually fooling themselves: most (usually waterfall) projects with precise up-front estimates later discover that those estimates are not accurate.

One of the strengths of story points is that everyone (including the customer) REALIZES that they are rough and don’t correspond to a precise delivery date — something that can be difficult to explain for estimates expressed in hours.

The first issue is to decide your definition of a “public” site. Simple solutions like “anything over SSL is private” fail in both directions: some content which is not private gets served up over SSL (generally a good thing), and some content which should be private gets served up over unsecured connections (like Facebook pages). I propose the following rule instead: any page which is seen with exactly the same content by at least N different individuals will be considered public. (The value of N can be debated, but something like 6 or 9 could be a starting place.) This means that any pages that personalize to each user (even just a “Hello Janet Smith” at the top) will not get shared — that may well be exactly the behavior that is desired. And it means that any page that IS shared must at least be known to a reasonably large group (so it’s not THAT secret). This definition works reasonably well for things like Google Docs (stays private unless it is widely shared) but works poorly for things like a corporate intranet (may get shared unintentionally if too many people use the tool). Perhaps a configuration in the tool could allow certain domains to be excluded from sharing; even more likely is that no company with serious security concerns will allow a tool like this to be run on their intranet anyway.

If we accept this definition for when pages should be shared, what we have left is just a technical problem (albeit a difficult one): how can we determine whether the exact content of a certain page has been seen by at least N other users without keeping a record anywhere that it has been seen by any individual user. Storing information is not difficult; we have already accepted that this project will use a P2P storage network and use some local storage so we can store information locally or inject it into a P2P network (perhaps a different network than the one used for document storage). But some obvious approaches won’t work. We can’t store a record of what has been seen on the local disk because that would record the entire browser history. We can’t store it in the P2P network in a form that can be retrieved because then we have already made the page public. And we can’t store it tied to a particular user-id (even an anonymous one) because tying together different pages visited is an effective way to identify an individual.

I will motivate the final solution by showing a series of partial solutions, each of which gets closer to solving the problem. As a first pass, we could use a P2P network which manages a large distributed hash table in which we can anonymously store values (multiple values for a given key) and which allows us to anonymously query the values. When a user viewed a page, they would use its URL as a key, and store a hash of the page as one value for that key, along with a counter. If the counter had already reached N then the page was public and program could go ahead and store the value in the storage network. The contents of non-public pages are not stored (locally or in the network), only their hash value. Old values would be kept in the P2P network for a good long time — a month or so would be good, to be sure of capturing the “long tail” of rarely viewed content, but would eventually be cleared out to make space for newer content.

This fails because there may be malicious participants in the P2P network. As soon as a malicious participant saw a value being inserted with a count of 1 they could immediately insert it again N-1 times. Or perhaps they could even lie and return a count of N even though the count was actually 0. In either case, users would be tricked into storing content that had not actually been seen by N distinct users.

To protect against the malicious participants we need to clarify our threat model. At this stage (where we determine what content to make public), a malicious participant is one who is trying to force the user to reveal content that shouldn’t be public. Since public is defined as “seen by more than N individuals”, a malicious participant who already knew the content could just publicize it directly (or push it into the storage network), so it is malicious participants who do not know the content of the page that we are concerned with. That leads immediately to a solution: instead of trusting the count, we can store something that only a user who had actually seen the page could generate: a “proof” that the page has been seen.

Simply create N fixed blocks of text. (It could be as simple as “This is block <N>” — the content doesn’t really matter.) To obtain proof number n, concatenate the page text with block n then take the hash of it. A user will query the P2P network for all hash values stored for a certain URL. Some of them may match the hash codes for proofs 1 through N. If all of those hashes are present, then N other participants have seen this value and it should be pushed into the storage network; if not then the lowest-numbered missing value should be added. A malicious participant, not having the content, cannot generate any of these values, and they cannot determine anything from the hash values themselves (since a hash cannot be inverted).

There is, unfortunately, one problem remaining. If a single user views the same content N times, they might push all N proofs into the network all by themselves. The values cannot be tagged with an ID of the individual who found it because that would allow multiple page views to be tied together, leaking identity information. And they cannot keep a record of which sites they have visited as that too would reveal information. The solution I propose solves this problem but replaces the certainty that there were N viewers with a probabilistic solution.

When the application is first installed it will select a random “user id”. This is just a string which is used to identify the installation so we can be sure not to double-count it. There needs to be a way for the user to read the “user id” and to modify it. That way someone who frequently uses several different computers can set the user id to the same value on all of them and will not double-count views just by moving between home, work, and mobile locations. Of course, it is important that we do not link this user id to the view history.

Now, instead of creating N different blocks to concatenate with the page contents we will create K different blocks (where K is somewhat less than N). For instance, K=8 is a reasonable value; certainly we require that K is much smaller than the number of users of the crawling system. As before, query to see which of the K proofs are present in the P2P system. The system takes a hash of the user id, mods by K to select a random (but consistent) value, k (where k is in [0..K-1]) essentially dividing all users up into K groups. If the proof for k is present, do nothing: it is possible that this proof was added by this same user. If the proof for k is missing, then add it. And if the total number of proofs is more than some threshold n, then enough people have seen the page and it can be published into the storage network.

What should n be? Well, malicious participants cannot affect anything since they don’t have the page contents and can’t generate the proofs. Each actual viewer of the page has what is essentially a randomly chosen value for k. So as people view the page, they are selecting a random value of k (from 0 to K-1). This continues until n distinct values have been chosen (some may have been chosen multiple times). We observe the count of distinct values seen so far (call it m) and want to estimate the number of actual page viewers.

I’ll skip over the math and jump right to the answer: the expected number of actual viewers is:

where Hn is the n’th harmonic number:

So if we use 8 for K and we see 4 entries, that means there must have been at least 4 users seeing the same content and there was probably about 6. If we see 5 entries then there must have been 5 different individual seeing the same content and there were probably 306/35 or about 8.7. The values required can be adjusted to reach the desired value for expected number of individuals seeing the page, while keeping K small (if it is large enough that the number of participants in on bucket is small then it can potentially be used to help determine your browser history).

Well, that’s enough essays on this topic for now. If you want to learn more, you may want to check out https://github.com/titanous/constantcrawl where there may eventually be an effort to build this.

There are many different types of P2P networks for storing an retrieving files. Different networks and protocols have been designed for different purposes and thus have different strengths and weaknesses. For instance, Freenet is designed to store and retrieve files with a focus on extremely strong anonymity guarantees and resistance to files being deleted, but with weaknesses such as being particularly slow. The Gnutella network is decentralized and efficient but the usage is not anonymized. Rather than fully specifying the design of a storage network, this essay will just attempt to list the requirements. Re-using an existing protocol (or even an existing network) might be the best approach; next best would be to design one by combining well-tested components from existing successful networks.

The P2P storage network would need to support the following functions:

1. Storing values by key with some metadata. The key would be fixed and known (the URL of the content). The metadata would be small — things like the date the content was viewed and the hash of the content. The value would be the content of the URL and would not be small — it would be the size of a web page, image, PDF, or whatever was being stored. (Content larger than a certain size could be rejected.)

2. Storing multiple values by the same key. A page may change or may be viewed differently by different users. The storage network needs to support storing these multiple values.

3. Querying the metadata of a certain key. In particular, we need to query to find out whether there already exists an entry for a given URL (key) with a particular value for the hash of the content. To prevent bad actors from responding “yes” when it is really absent we may need to return the value. We would expect this query to be by FAR the most common query performed — hundreds of times more frequent than any other function the storage network would perform.

4. Retrieving the value (and metadata) for a given key. If multiple values were stored we would either return all of them or select one by metadata.

5. Querying to find out what keys had new values stored within some recent time period. This would not be used by normal users, but by organizations like the Internet Archive or Duck Duck Go that actually wanted to download the contents of this storage, thus making it into a means for crawling the entire web. It would be acceptable if the only way to perform this query were to contribute significant resources to the P2P network and also if the query were likely to be accurate instead of certain to be accurate.

Those are the only functions that would be required, but there are some broad characteristics of the P2P network that are also essential:

A. Anonymity: it should be quite difficult to determine who is performing any of calls 1 through 4. (It is OK if call 5 is not anonymous.) Otherwise, the network could be used to determine a user’s browsing history. This could be accomplished via onion routing, the elaborate approach used by Freenet, or any other solution that makes it impossible for any participant in the network to tell for sure what other members of the network are searching for.

B. Legal Protection for Stored Content. Legal liability for storing content on a P2P network can be a genuine problem. But in essentially all jurisdictions this can be avoided if the individuals participating in the network are not able to determine just what content they have. For instance, some systems break each file up into chunks which are encrypted so they cannot be understood without possessing all chunks, then store different chunks in different locations. No one person has any file on their system.

C. Reliability. The P2P system must retain its data even if peers join and drop out with some regularity. This means keeping redundant copies of everything. It must also function if some minority of the members of the P2P network are antagonists who will abuse the protocol in an attempt to harm the network. These are standard features on most P2P networks.

D. Purging of old values. The size of the P2P storage network cannot simply grow forever. So values need to be evicted eventually. Evicting values that are older than a certain age should work fairly well — if the content is still being viewed frequently it will be re-loaded rapidly and by keeping for a certain amount of time we would provide an opportunity to download the content for those using this to crawl the web. Also acceptable, but perhaps less ideal, would be automatically evicting the least frequently requested values.

That’s about it for requirements for the storage network, except that given these constraints we would like for it to be as fast as possible, especially for frequently requested values. This implies that the number of network hops should be minimized (despite this, anonymity will require a fairly large number) and may favor solutions that store frequently-requested information in multiple locations.

The next (and final) installment of this series will discuss the interesting (and challenging) question of how to tell when a viewed page can be made public.

The basic experience would be a perfectly normal browsing experience: users would launch their favorite web browser normally, would browse around the web normally and everything would “just work”. This means that clearly the system would function as a browser plug-in. Fortunately, nearly all modern browsers support some form of plug-ins. In principle, one could also develop this as a proxy, but it would be much more difficult to develop an effective UI.

An easy-to-use installation process is important if one is seeking a large user base. This means using the normal means for the platform (an installer for windows, but RPM, yum, etc for linux). It means that the installer sets up the browser plugin, allocates the disk space needed for storage, and creates the services needed to join the P2P network.

The plugin itself offers two basic pieces of functionality. One would be that it captures the content of the web as it is viewed, and (where appropriate) archives it for the crawl. The other is the benefit for the user of the plugin: it allows them to view content from the crawled archive when the normal site is slow or unavailable. (For instance, the “slashdot effect” where a smaller site is featured on a popular news site like Slashdot or Reddit and becomes overwhelmed.)

The plugin should have three basic “modes”. Any well-behaved plugin should provide an easy way turn it off, so one mode is “disabled”. Another mode would be for loading all content (or perhaps just re-loading the current page) from the archive. And of course there would be the normal mode (more on this in a moment). The mode affects the page rather drastically (changes where we are getting it from or whether we are potentially sharing it with the world) so the plugin should probably provide an indicator of some sort in or near the URL bar, and this indicator might as well provide the means for switching the mode as well.

What behavior would we want in “normal mode”? Pages that get viewed are eligible for sharing, but only if that page is determined to be a “public” page (see other essays for details on this). So the plugin would need to capture the content of the page and immediately after rendering it (perhaps in a separate thread) begin to process it for possible sharing. I’ve used the term “page”, but essentially all content should be treated this way, including images, CSS and JavaScript files, even AJAX calls: any content downloaded by the browser.

The next question is when content should be downloaded from the archive. Unlike Google Web Accelerator, I think it is unlikely that this design involving anonymous P2P technology will ever be faster than ordinary browsing to a normally functioning web site. But it can be available in those cases where the ordinary site no longer is, where the HTTP request times out, or a 404 (page not found), 410 (page gone), 503 (server overloaded) or some other error is returned. The simplest solution would be to attempt to load the page from the storage network whenever these conditions occur. Always trying to load every page from both web and storage (and displaying the one that arrives first) would put far too much load on the storage network for pages that would never be viewed.

It is worth noting that this storage network can store multiple versions of each page. This could be because the page has changed over time, because it is served up differently to different classes of user (perhaps by geographic region), or for stranger reasons like a malicious user injecting false versions of a page into the network. This property might lead to some very interesting and powerful uses (“See old versions of any page!”) and it might pose new technical challenges (“Allow trusted entities like the EFF to somehow flag which version of the page is ‘real’”). Such considerations are an excellent subject for future consideration, but the next essay will address technical challenges with the storage and the final part will show how to anonymously determine whether to share a page.

Well, Google eventually dropped Google Web Accelerator (I wonder why?), but the idea is interesting. Suppose you wanted to build a similar tool that would capture the web viewing experience of thousands of users (or more). For users it could provide a reliable source for sites that go down or that get hit with the “slashdot” effect. For the Internet Archive or someone a smaller search engine like Duck Duck Go, it would provide a means of performing a massive web crawl. For someone like the EFF or human-rights groups it would provide a way to monitor whether some users (such as those in China) are being “secretly” served different content. But unlike Google Web Accelerator, a community-driven project would have to solve one very hard problem: how do this while keeping the user’s browsing history secret — the exact opposite of what Google’s project did.

This topic came up at a meeting of the Philly Startup Hackers group, and after an entire evening of vigorous discussion, we think that such a project would be technically feasible. In this series of essays I will attempt to outline the technical architecture of this solution. This first one will explain the major components and how they fit together.

Broadly, I’ll describe three different problems to be solved and we’ll assume that the solution to each one is a layer in the architecture. Problem (A) is the user interface, problem (B) is deciding what information is public (surprisingly, this turns out to be the most difficult part), and problem (C) is storing the pages.

The solution to problem (A) (user interface) is quite straightforward. That is not to minimize it: implementing the user interface well is by far the most work of the whole project and the piece most likely to contribute to the success or failure of it. But the approach to take is clear. This should integrate into the customer’s browser, and with modern browsers that means implementing it as a browser plug-in. Also essential to the user experience is the installation experience: to be successful, this needs to be extremely easy to install and very simple to configure (preferably with no configuration required for use).

Problem B is to decide what pages should be public, and which are private to the user. The UI can help here if users can easily flip into modes where everything is captured or where nothing is. But one cannot expect the user to click something before (or after) every page — there also needs to be a “normal” browsing mode. We’d like to (anonymously) record the majority of pages visited (after all, that’s the point of the tool), but a page showing your bank account probably shouldn’t be shown, nor should a Google Docs essay you’ve been writing. Assuming that everything viewed with HTTPS is private and everything viewed with HTTP is public seems much too simple a rule, particularly as privacy sensitive sites are beginning to default to HTTPS for all users.

So the approach I am proposing is that we assume that if several people see the exact same page then it must be a public one. My bank’s logged-in view of my accounts won’t be seen by any other users, while the Google Doc essay I share with a friend will only be seen by a couple of people. If we set the threshold to something like 6 or 9 users then we can be fairly confident that the content was public. To capture rarely-seen sites we’d want the count to last for some time: 6-9 users within the same month, perhaps. Now the technical challenge is to figure out how to tell whether several people have seen the content without revealing it (since it’s private) and without leaving any trace that we have viewed it (for privacy reasons).

Problem C is storing the content. Spotify is a popular music player which has been installed by millions of users. Yet they don’t need huge servers to transmit all those data streams. Instead, they use P2P technology and each user provides a certain amount of storage and a certain amount of bandwidth. Other projects like Freenet have proven that P2P sharing can store data and keep all the participants anonymous. So I propose leveraging fairly standard P2P approaches (or better yet, an existing P2p storage network) for storing, finding, and retrieving the content.

Well, that’s enough for one essay. Check back in part 2 for more details about the UI, part 3 for a discussion of the storage network, and part 4 for analysis of how to anonymously determine whether something can be shared.

If you ever see “Host error number XXX”, it means that this was the XXX’th error of the day that this Profile instance wrote to the logs. Get someone to look it up in the Profile logs.

Also, Calling mrpc ZWRAP with [925, 8864, ""44758220"", |!|] will fail if 8864 is not a valid profile userid (which is the case for me).

1. Launch PowerPoint (these instructions work for Office 2010).
2. Open the presentation to be fixed.
3. Go to the File menu, and select “Options”.
4. Select “Customize Ribbon”.
5. On the right-hand side of the complex dialog find the “Main Tabs” section. Check the checkbox next to the “Developer” tab.
6. Click “OK” and return. You should now have a “Developer” menu above the ribbon.
7. Select the “Developer” menu to display the developer ribbon.
8. On the ribbon, click the button labeled “Visual Basic”
9. In the upper-left-hand side of the screen there is something labeled “VBAProject” with sub-folders labeled “Modules” some of which have entries with names like “Module1″. You may have to expand the tree-view widget to see some of these.
10. For each module, double-clicking will open it. Those that are completely blank (all of them in my company’s template) are useless and can clearly be deleted.
11. Delete a module by closing it (if you had opened it), then right-clicking on the module and selecting “Remove Module1…”. It will offer to save, but you won’t need to do that.
12. After doing this for all unwanted modules, go to the “File” menu and select “Close and Return to Microsoft PowerPoint”.
13. Save your newly-changed document.

By the way, in case anyone couldn’t tell, after experience with it I really hate Microsoft’s “ribbon”. The old approach “Menus” required people to look around through lots of menus to find the commands they needed (although if they used a command frequently, they could read the menu to see what keyboard command would execute that). Power users could modify the menus if they wanted to (but hardly anyone did). In the new “Ribbon” interface, the commands that people use a lot are just one click away — as long as you know what arcane icon represents the action that you want to perform. If you need to find a new command you no longer have to look through the menus to find it… instead you simply perform a web search to find someone else who ran that command and follow the arcane set of clicks that they wrote in order to locate the mysterious and well-hidden button that performs the action. Power-users are people who know how to perform an action without looking it up on the web.