Posts

Showing posts from 2020

We only have models, not knowledge

We know that water boils at 100 degrees Celsius, right? Or not ; on mount Everest the water boils at around 70 degrees Celsius, so you can't even boil an egg . Ok, we have a more complex model that links the boiling temperature with pressure, but even that model is not exact at any given pressure. On some temperatures, our linear models fail, for example water cooled below 4 degrees Celsius will not continue to get denser, but it will actually expand, allowing the cold water to float on the surface until it becomes ice. In general, lower temperature would predict lower density in substances. There is nothing in the common physical laws that would predict such anomaly  in water, while we can later construct even more complicated models to explain what is happening with water.

Unequal inflation

It's common knowledge that if you print more money, you will produce inflation. However, things are more complex than that. You can have both inflation and deflation in the same time. More often, we have relatively low inflation on common products and high inflation on specific products - think real estate. Money velocity First, there is that "money velocity" that equals the equation Products * Price = Money * Velocity . The money velocity is intuitive, but cannot be practically measured except from this equation - thanks to my namesake friend for this idea.

3. Data Consistency and the "Theory of relativity"

"If a tree falls in a forest and no one is around to hear it, does it make a sound?" If a system has an inconsistency but no one is able to observe it, is it still an inconsistency? Introduction We need to scale data processing systems geographically to achieve lower Latency and (at least partial) Availability in case of network Partitioning. But CAP / PACELC tells us that we cannot achieve strong Consistency in this case. When we increase the Consistency requirements, we have to accept lower Availability and higher Latency. What is the minimum consistency level that we need? If eventual consistency is enough for your system, things are pretty clear and relatively simple. Most probably you want to achieve Strong Eventual Consistency that is relatively cheap and provides nice guaranties. For this you will have to you something like CRDT . Some theoretical results assure us that you cannot find something way cleverer than CRDT that achieves Strong Eventual Consistency. Often...

Religions are (Software) Operating Systems

It is hard to live without a religion. Each religion comes with its set of APIs, for blessing and cursing, for forking a process child and for being sure a dying process doesn't become a zombie. Religions come with their set of protection mechanisms and their corresponding violation errors and punishments.

2. Flow consistency - read-your-writes consistency

  2. Consistency, Availability and low Latency in Distributed system (workarounding the CAP/PACELC theorems)   Introduction: " 1. Cache and Data Consistency in Distributed systems (CAP/PACELC/CRDT) "   TL;DR Full strong Consistency in geographically Distributed systems can only be achieved by sacrificing Availability (per CAP theorem) and with prohibitive Latency costs (per PACELC theorem). However, we can still design consistent enough systems that continue to function when one geographical region is down and without paying the inter-region latency most of the time. While eventual Consistency is OK many times, there are still cases when we want a strong read-after-write consistency for certain read-after-write flows. There is an optimum design that assures strong Consistency inside read-after-writes flows . Arguably, this is the highest Consistency level that can be assured without a prohibitive impact on Availability and Latency.

Devil's offer

Imagine that humanity is offered the following deal: All living humans will enjoy 100 more years of healthy and happy life, finished by a peaceful death in sleep. Immediately after signing the deal, humans will forget the deal and will not worry about this "deadline". If you sign the deal, no human will even experience any suffering. As life will be extended to 100 years from now regardless of the starting age, no human will even witness the death of another human. Even old people would enjoy a healthy and happy life, becoming physically young again and regaining their mental capacity - without losing their experience. Each life will be full of accomplishments, not only by material welfare, but also with most spiritual accomplishments that a human can reasonably expect. Children will still be born, parents will still be happy about them - even more so, because all would go smooth with no worries and no accidents. People would live happily and without any worry until the ...

Morality and killing

Killing is morally wrong universally, for most people adhering to a moral system, right?  What about killing an aggressor that tries to kills hundreds of people in your community? Suddenly that killing to save the life of many innocents does not seem so repellent for most people, don't you agree? What if the killing of an aggressor is the only solution to save the life of a member of your family? Even many religious people would consider such defensive killing the right thing to do if there is no other solution.

The Capitalism theory

We have many anecdotal evidences that economical capitalism creates more welfare than economical communism. However, we lack a clear model to explain what is necessary for the capitalism to work and what is only contingent - not contributing or even affecting the welfare creation. Without such model, we are left debating with ideological arguments like "private property is a natural right" or the anecdotal argument that "communism never worked well enough". The opposing side would naturally reply with some other anecdotal evidence about the shortcomings of capitalism and the possibility that "communism failed because it was not correctly implemented". I will argue here that we can actually model the way private property positively impact the creation of overall welfare. On the other hand, we can also identify cases when private property can negatively impact the overall welfare. I use "common welfare" in the common, intuitive sense: th...

Evil as striving for a local maximum

Let's observe first that we tend to attribute evil when there is a sufficient capacity of understanding the consequences of one's actions. A small child that produces a tragical accident is not considered evil usually, because there was no intent in doing that harm. We then observe that we judge evil in the contrast to another, better, possible action: "he could have decided to not kill him". We don't attribute evil when someone produces harm without having the possibility of doing something better. Moral judgement on evil is therefore a judgement of one's actions and decisions, in relation to some other actions or non-actions that would have resulted in a better world. That better is a highest "local maximum" that we can envision. The misdirected action is usually dictated by another, more narrow,  goal that is aiming at a smaller local maximum. For example, one can aim his narrow, selfish, good (local maximum) at the expense of others. T...

1. Cache and Data Consistency in Distributed systems (CAP/PACELC/CRDT)

Abstract There is always a tension between data  C onsistency and system  A vailability when  P artitioning a system across datacenters (think  CAP ). Especially data cache-ing poses interesting challenges. This tension becomes way more acute as soon you have 2 data centers separated by more than 10ms latency. I present below some of the problems along with possible solutions. In the end I will present an elegant solution that maximizes  A vailability while providing the needed C onsistency level for read-after-writes flows. The solution requires the client to carry a monotonic  id  along the flow. I would postulate that any solution where the client don't carry some consistency info will provide a higher latency that the presented solution (see chapter "Flow consistency" ) . The examples below are simplified to be intuitive and easy to understand, however these learnings also apply to N datacenters. How it starts Suppose you started with on...

Hell

Being burned in hell for an eternity is not the worst that can happen. Even worse than this, for me, is to be entirely washed out from Existence. Burning in hell looks pretty painful indeed, but it still contains a glimpse of hope. After eons of suffering, a redemption is still possible, potentially. Suffering from burns is still a property of Life, something that suffers must exist and must somewhat live. There is still something that exists in the burning hell... Even worse than this, for me, is to disappear, gradually, then totally, from Existence. The eternal void is more frightening that any eternal pain. The eternal void does not leave any glimpse of hope for a better future. The eternal void is no future at all. * * *

On jobs and welfare

We often hear the expression "this creates jobs". But job creation cannot be good in itself. The state could create well payed jobs for everyone for doing nothing. Even with jobs, if nobody produces food, then people would starve. What is important is welfare creation - goods and services, that is often accompanied by occupied people that have jobs. Such welfare creation should equate to a better life for the ones who consume that created welfare. There are many nuances here. In these days, only a fraction of jobs is actually involved in creating goods and services that directly affect the quality of life. There is a high portion of jobs that are involved in organizing the welfare creation and redistribution. This includes management, state administration, banks, lawyers, police and more. Without a system to control the fair exchange of goods through money, people would tend to break contracts and steal from the welfare created by others. While such "administrative...