Posts

Showing posts with the label Computer Science

On Russel's Paradox

Russel's paradox is defining "the set of all sets that don't contain themselves". You can show that such construct leads to a contradiction. Read the wiki link for details. I started this article as answer to this article . There is another interesting paradox discussed by Bertrand Russell: "The smallest positive integer not definable in under sixty letters", actually better known as Berry's paradox  You can also prove that such number creates a contradiction, because you just defined it under sixty letters. For me, the above paradox shows that it is not enough to create a condition for something to exist with that condition - or a set of such elements. Such paradoxes argue that such objects defined by a predicate cannot exist sometimes.

Data Consistency in Distributed system - CAP, PACELC and more

A trilogy about Data Consistency and how to workaround the CAP and PACELC theorems: 1. First issues with multi-region and multi-cloud software scalability 2. Solutions to have enough Consistency without sacrificing Availability and Latency 3. Theoretical arguments on the nature of CAP/PACELC problem and why you cannot find a better solution than the above one Dear reader, please leave a message if you exist! ;)  Also, please share this article if you find it interesting. Thank you.

3. Data Consistency and the "Theory of relativity"

moved to  https://software.mihvoi.ro/2026/04/3-data-consistency-and-theory-of.html "If a tree falls in a forest and no one is around to hear it, does it make a sound?" If a system has an inconsistency but no one is able to observe it, is it still an inconsistency? Introduction We need to scale data processing systems geographically to achieve lower Latency and (at least partial) Availability in case of network Partitioning. But CAP / PACELC tells us that we cannot achieve strong Consistency in this case. When we increase the Consistency requirements, we have to accept lower Availability and higher Latency. What is the minimum consistency level that we need? If eventual consistency is enough for your system, things are pretty clear and relatively simple. Most probably you want to achieve Strong Eventual Consistency that is relatively cheap and provides nice guaranties. For this you will have to you something like CRDT . Some theoretical results assure us that you cannot fin...

2. Flow consistency - read-your-writes consistency

 moved to  https://software.mihvoi.ro/2026/04/2-flow-consistency-read-your-writes.html 2. Consistency, Availability and low Latency in Distributed system (workarounding the CAP/PACELC theorems)   Introduction: " 1. Cache and Data Consistency in Distributed systems (CAP/PACELC/CRDT) "   TL;DR Full strong Consistency in geographically Distributed systems can only be achieved by sacrificing Availability (per CAP theorem) and with prohibitive Latency costs (per PACELC theorem). However, we can still design consistent enough systems that continue to function when one geographical region is down and without paying the inter-region latency most of the time. While eventual Consistency is OK many times, there are still cases when we want a strong read-after-write consistency for certain read-after-write flows. There is an optimum design that assures strong Consistency inside read-after-writes flows . Arguably, this is the highest Consistency level that can be assure...

1. Cache and Data Consistency in Distributed systems (CAP/PACELC/CRDT)

moved to:  https://software.mihvoi.ro/2026/04/1-cache-and-data-consistency-in.html Abstract There is always a tension between data  C onsistency and system  A vailability when  P artitioning a system across datacenters (think  CAP ). Especially data cache-ing poses interesting challenges. This tension becomes way more acute as soon you have 2 data centers separated by more than 10ms latency. I present below some of the problems along with possible solutions. In the end I will present an elegant solution that maximizes  A vailability while providing the needed C onsistency level for read-after-writes flows. The solution requires the client to carry a monotonic  id  along the flow. I would postulate that any solution where the client don't carry some consistency info will provide a higher latency that the presented solution (see chapter "Flow consistency" ) . The examples below are simplified to be intuitive and easy to understand, however the...

Linux Ubuntu 19.04 full install on USB stick - fix UEFI/Grub

Image
Even better reads: Data Consistency in Distributed System - CAP, PACELC My  Secular Philosophy - Life, God, Knowledge, Morality, Ethics This blog is mainly focused on my Philosophical and theoretical ideas. From time to time I have practical posts that are too few to deserve a blog by themselves. I still hope my other work (like above) is even more valuable than such practical article as below. * * * This article explains how to install Ubuntu 19.04 or similar on USB+UEFI, like you would do it on a hard drive. Regular USB install will likely fail, you need the fix below. Ubuntu installer has a bug when installing on USB using a PC that already has an UEFI boot system (like Windows 10). Such thumb-drive is not only persistent, you can also do regular updates on it, customize the programs, repair the file-system. This is more than just a live "USB Pen Drive Linux" with persistence, it is your full Linux system in your pocket. Only (U)EFI boot is covered here, but most recen...

Linux Grub/UEFI boot system - introduction / howto for Ubuntu 19.04

Better read:  Data Consistency in Distributed System - CAP, PACELC This blog is mainly focused on my Philosophical and theoretical ideas. From time to time I have practical posts that are too few to deserve a blog by themselves. I still hope my other work (like above) is even more valuable than such practical article as below. * * * On short : in order to Boot Linux in UEFI mode: UEFI loads Grub and a minimal grub.cfg from a fat32/EFI partition. Ubuntu uses 2 directories: EFI/Boot and EFI/ubuntu Grub detects the root partition (via UUID) and loads the kernel and initrd from that Linux/ext4 partition. In Ubuntu, the minimal grub.cfg calls another grub.cfg from the ext4 partition - containing boot menu. More details

Why Kolmogorov complexity is not computable - root cause

Please check my informal introduction to "Kolmogorov complexity" if you are not familiar with the subject. In this article I will approach some less discussed ramifications of it and hopefully some new ideas: an example of a Kolmogorov compression that would require an arbitrary high amount of working memory before shrinking to the compressed string (conjecture). 1. Root cause why "Kolmogorov complexity" is uncomputable Kolmogororov complexity is uncomputable because Turing machines have a halting problem :) Let's start with the observation the the Halting problem of a Turing machine having a limited band is actually decidable, unlike the infinite band one. A "finite band Turing machine" is actually a finite state machine that will either stop without going through the same state, or will cycle infinitely. Having a limited number of possible states, it cannot run too much without repeating a previous state. As soon as the machine returns to...

Kolmogorov complexity - short, informal introduction

I want to give you an intuition about  Kolmogorov complexity , without being very formal. Basically, Kolmogorov complexity defines a measure of how small can you compress a certain string of characters. You can think about it like measuring how small can you make a file (string of characters) by compressing it with ZIP or similar. A file storing a very redundant information (like "aaaaaaaaaaaaaaaa....") can be compressed in a much smaller size compared with a random one. For calculating the "Kolmogorov complexity", the "compressed" form is considered to be something like a self-extract archive: the shortest program that can output the initial string of characters (the "file/string to be compressed"). The Kolmogorov complexity is more general and includes classical compressing algorithms. For example a Huffman code is not very powerful in compressing a string like "123456789101112131415161718192021222324..." (concatenated consecut...

P=NP is undecidable (conjecture)

" P equals NP? "  is a million dollars unsolved problem . Introduction " P" is the class of problems where we have algorithms that solves the problem in polynomial time. " NP"  is the class of problems having solutions that can be proven in polynomial time once you "guess" the solution. This is analogous with the sorting problem: it's easier to check the correct sorting of a list than to actually sort a list. Of course, if you can fully solve the problem in polynomial time, you can also prove it in polynomial time, so P is included in NP. However, NP seems to contain some very hard problems that cannot be currently solved in polynomial time. It is believed that NP contains some problems that can never be possibly solved in P. I will not do a formal proof, more like an argument or a sketch. You might have more patience and skills to develop some of these ideas even if they might have flaws in the current form. You can take it a...

Simple patterns of intelligence

This is an idea about how intelligence could emerge from simple biological patterns. If confirmed, this could have direct impact about how we understand the human brain and how we can design new Artificial Intelligence systems. There is now a widely spread belief that human brain intelligence is based on very simple structures (neurons) that can implement various thinking processes, based on location and structure. This belief is sustained by researches that shows how brain can learn to perceive with regions that were not initially related to that sensing type (like " seeing with your tongue "). This is the approach taken by various researches that are trying to reproduce human-like brain processes  (like recognizing objects) with general purpose neural networks.

Can a computer (or human) simulate itself?

A virtual machine is a common example of simulation. It can take the description of a (virtual) "computer" state and continue the execution from that point on. You can simulate a huge computer given a computer with more memory than the virtual machine, although it might be way slower. However, such virtual machine does not answer the problem because it normally simulates a computer with less memory. The question is, can a computer to simulate a virtual computer that has the same capabilities as itself? Before answering this, let's exclude some trivial (forbidden) cases of "simulations": A calculator will always do exactly what it does, it will be identical with itself, we will not call this a simulation of itself. Copying the exact state of another identical computer in a "twin brother" will not be considered simulation. Starting 2 identical computers with the same input will not be considered like one is simulating the other.

Shortest program that prints the play "Othello" by William Shakespeare

We have this requirement: "Create the smallest program that will print the full text of the play Othello by William Shakespeare". I should warn you that the requirement hides a trick and you will hate the solution. The program does not need to run efficiently. Actually, it does not need to run in reasonable time. It just has to be very short. And to be proven to answer the requirement, eventually... No external random source is allowed and no input from user.

Sorting is computationally harder than playing chess

I will argue below that sorting of a big array, using only comparison, is inherently harder than solving the problem of choosing the best next move on a chess game. One could say that it is not fair to put in competition a big, however finite, problem like chess with an arbitrary big problem like sorting n elements. Fair enough; the interesting thing is that sorting becomes inherently harder than chess for an "n" that is way smaller than you would expect, like sorting 3000 elements. At least in theory... Sorting takes a minimum of n* log 2 (n) comparisons,  more exactly  log 2 (n!) This is about sorting "by comparison only" and can be proven in 2 step. You can skip to the "chess" part if you already know this. 1. Sorting can be done in  log 2 (n!) comparisons, that is just slightly less than n* log 2 (n). We are used to consider sorting an O (n* log 2 (n)) problem. Actually there are algorithms that can do the sorting with little less t...

Mathematical incompletness and life

A friend told me once that we are living in an Universe full of unprovable truths. He was referring to the Gödel's incompleteness theorems . Without being very formal, this says that there are many mathematical propositions that might be true but we cannot prove them using a computer program or any predictive way of checking, no matter how huge amount of time we can spend. We can find the proof for some, by luck, but for most of them we cannot find either a proof or disproof. In fact this "unprovable" propositions are way more many than the provable propositions. The provable propositions are like the set of natural numbers "N" and the unprovable propositions are like the set of real numbers (R).

When float does not change when incrementing - C/C++ programming

Better read: Data Consistency in Distributed System - CAP, PACELC Morality series - a heptalogy This blog is mainly focused on my Philosophical and theoretical ideas. From time to time I have practical posts that are too few to deserve a blog by themselves. I still hope my other work (like above) is even more valuable than such practical article as below. * * *   What would you expect if you increment 20 millions as float? Do you expect that it will not change at all? Check the output of the simple program below. You may know the explanation, but did you expect it to happen to such relatively small values? Output: user@ubuntu$ gcc float_inc.c ; ./a.out float has 32 bits a=20000000.000000 a+1=20000000.000000 a=16777216.000000 a+1=16777216.000000 a=16777218.000000 a+1=16777220.000000 a=16777220.000000 a+1=16777220.000000 a=16777220.000000 a+1=16777220.000000 a=16777220.000000 a+1=16777220.000000 a=16777222.000000 a+1=16777224.000000

Intuition and neural networks

Human intuition behaves similar to a neural network software algorithm: it produces an estimation of a certain probability, based on prior experience. The same with an artificial neural network, human intuition needs to have sufficient training experiences on that field so it can extract regularities. The training experiences should be able to provide adequate and rapid feedback that can be used to adjust the internal model, for both biological and software neurons.