'Every UUID V4' can list and search all UUIDs, but what technical challenges did it face?

Every UUID V4
https://everyuuid.com/

Writing down (and searching through) every UUID · eieio.games
◆Rendering
Because 'Every UUID V4' was implemented as a web application, browser limitations sometimes directly translate to application limitations. As a way to display a huge number of UUIDs, Nolen initially considered 'creating a very long div element, placing the UUIDs in it, and rendering only the portion that would be displayed on the screen.' Unfortunately, however, there is a limit to the scroll position that browsers can grasp, and some browsers store it as a 32-bit integer, which is completely insufficient. Even if the scroll position could be stored as a 64-bit integer and the area for displaying a single UUID was limited to 1 pixel, it turned out that an enormous amount of scroll space would still be needed. When he asked ChatGPT for a solution, he was told 'it's impossible.'

As a final solution, Nolen abandoned the idea of scrolling within the browser and instead decided to fix the screen height to the browser height and manage the 'virtual scroll position' using
◆ Ordering
Once the rendering issues were resolved and we could actually see the list of UUIDs, a new problem arose: 'The UUIDs are not sorted in order of interest.' The expression 'in order of interest' is difficult to understand, but to quote Nolen, 'We want to see UUIDs like bb166283-2e09-4b72-ba32-70a43521c70e, and certainly not something like 00000000-0000-4000-8000-000000000000.' The ordering constraints that Nolen devised are as follows:
- There must be no identifiable pattern in the order of the UUIDs.
- Display all UUIDs only once.
- Always display UUIDs in a consistent order.
Translating the above constraints into programming terms means that 'integer indices need to be mapped to UUIDs in an 'interesting way'.'
The first method considered was pseudorandom number generation using the Linear Congruential Grouping (LCG) method . However, it was found to have problems with the randomness of the lower bits, and more importantly, that the intermediate steps became extremely large with large constants, making it difficult to perform calculations that 'skip steps,' so this method was not adopted.
The next concept considered was 'entropy and bijectivity.' The constraint 'display all UUIDs exactly once' can be mathematically interpreted as a bijective mapping, meaning that each index corresponds to exactly one UUID and vice versa. As Nolen delved deeper into bijectivity, he realized that 'bijectivity is maintained if the operation of adding entropy is reversible.' The approach devised based on this insight is as follows:
- Input data is 122 bits (number of bits excluding the UUID v4 version number and variant bit).
You may do anything to increase the entropy of a bit if the following conditions are met.
Ultimately, you will obtain exactly 122 bits, the number of bits required to create a UUID.
- The original 122 bits can be restored from the final 122 bits.
To achieve reversible scrambling and entropy addition, encryption using the Feistel structure was adopted. The Feistel structure has a structure in which the input is divided in half into two chunks, a round function is applied to one chunk, and a new chunk is created by XORing it with the other chunk, and this process is repeated. The remarkable thing about the Feistel structure is that it is reversible no matter what function the round function is, as long as it is a pure function .
Mapping the scrambled bits obtained using Feistel encryption to the UUID v4 format proved to be an extremely difficult task. Because UUID v4 includes version bits and variant bits according to the specification, it takes the following format:
[code]
UUID V4 format. Values:
X: any 4 bits
4: These 4 bits must be 0100 (4)
V: the top two bits here are 10; valid values are 8, 9, A, B
XXXXXXXX-XXXX-4XXX-VXXX-XXXXXXXXXXXX
[code]
To ensure all bits matched the specifications, we had to carefully map the 61-bit left and right chunks from the Feistel structure to the bits required for the UUID structure. This 'bit manipulation' was buggy and extremely difficult, but ultimately we obtained the desired 'very good list of UUIDs that appear random but are in a consistent order.'

◆Search
While the goal of sorting UUIDs by interest was achieved, another problem arose. Because the UUIDs appear to be ordered randomly, finding the desired UUID from the enormous list became difficult. Since the mixing function is reversible, adding a function to search for a specific UUID was easy; it simply involved calculating the index from the UUID and jumping to the position indicated by the index. However, the situation changes when it comes to accommodating the request to search by substring. While this is not a problem for UUIDs within the display area, targeting all UUIDs would require storing trillions of items in memory, which is not practical.
Our initial approach attempted to generate patterns in which the search string within the UUID can be placed, specifically patterns that take into account hyphens, version numbers, and variant bits.
[code]
# valid patterns for '4AAA-'
XXXX4AAA-XXXX-4XXX-VXXX-XXXXXXXXXXXX
XXXXXXXX-4AAA-4XXX-VXXX-XXXXXXXXXXXX
XXXXXXXX-XXXX-4AAA-VXXX-XXXXXXXXXXXX
[code]
Next, we attempted to determine which bits of the input index needed to be set in order to generate a scalable pattern. However, this method failed because the dependencies of the bits output by encryption were far too complex.
The final compromise that was implemented was as follows:
- Generate multiple valid UUIDs based on the search string pattern.
- Obtain an index from the generated UUID
- Select the UUID closest to your current location based on the index.
While this method isn't perfect, it allows users to sequentially trace UUIDs that match substrings.
◆Future Outlook
Nolen is pleased that he was able to consolidate all UUIDs into 'Every UUID V4,' but believes that the social features (such as a 'like' function by friends, a UUID sharing function, and a function to introduce currently popular 'trending UUIDs') should be further enhanced. At the same time, he is interested in exploring more efficient search methods for UUIDs that are arranged in a random order. The development of 'Every UUID V4' has highlighted the challenges of accessing and searching a vast number of UUIDs.
Related Posts:
in Software, Web Application, Posted by log1c_sh







