Posts

I never had the chance to understand how IPFS – the magical public network where a bunch of Web3 stuff is stored – worked until now. Now that I finally have the chance (and the excuse) to look into it, however, I can attempt to answer a number of burning questions I had: 1. who pays for storage in IPFS? Is it free for users or what? 2. How does the process of storing and retrieving a file actually work?

Open membership P2P systems are plagued by security issues and DHTs are no exception. In this post, I will try to summarize my understanding of one attempt at mitigating some of the security issues found in the Kademlia DHT in adversarial environments. The resulting work is named S/Kademlia, and it is significant in that it underpins a number of practical DHT implementations, including the libp2p DHT spec.

As professional tides bring me into P2P once again and I revisit some classics of the P2P literature, it is disconcerting to realize just how poorly did I really understand some of those things in the past. And one of those things is the expected lookup complexity of a DHT like Chord.

Distributed consensus - i.e., the problem of getting a set of processes to agree on a certain action or value over a network - is a fundamental problem of distributed computing. It arises in settings that range from group membership (i.e., tracking the set of servers in a server group that are currently online) to distributed database commits. And yet, despite its pervasiveness, it is anything but simple to solve.

I have been doing some Ruby programming as part of my recent work ventures, and I think a nice way to learn a new language is trying to understand how to map your previous way of doing things into this new environment, and discovering new things along the way. Having been programming in Python for the best part of the last 4 years, I got used to as a convenient way of structuring certain computations.

Histograms are probably one the most intuitive ways of performing density estimation in numerical data. In this writeup, I discuss some of their deeper meanings and properties, as I try to digest Wayne Oldford’s excellent lecture on the subject. In particular, we look at how well one can expect to approximate an underlying density as one has access to more samples and utilizes narrower bins. We then discuss a form of histogram that is much less sensitive to the location of bin boundaries, called an Average Shifted Histogram [1].

Backpropagation is a method for computing gradients of complex (as in complicated) composite functions. It is one of those things that is quite simple once you have figured out how it works (the other being Paxos). In this document, I will try to explain it in a way that avoids the issues I had with most of the explanations I found in textbooks and on the Internet. Inherently, this means that this document is well-suited to my way of thinking, but my hope is that it may also suit yours or, at the very least, may give you a different perspective as you try to grasp how backpropagation works.