|
| 1 | +# ❄ Spaceflake ❄ |
| 2 | + |
| 3 | +[](https://pkg.go.dev/github.com/kkrypt0nn/spaceflake)    |
| 4 | + |
| 5 | +A distributed generator to create unique IDs with ease; inspired by [Twitter's Snowflake](https://github.com/twitter-archive/snowflake/tree/snowflake-2010). Blog post about this project can be found [here](https://krypton.ninja/2022/11/08/Generating-unique-IDs-with-the-Snowflake-algorithm/). |
| 6 | + |
| 7 | +## What is a Snowflake? |
| 8 | +Apart from being a crystal of snow, a snowflake is a form of unique identifier which is being used in distributed computing. It has specific parts and is 64 bits long in binary. I simply named my type of snowflake, a **Spaceflake**, as it does not compose of the same parts of a Twitter Snowflake and is being used for [Project Absence](https://github.com/ProjectAbsence) and other projects of myself. |
| 9 | + |
| 10 | +### Structure |
| 11 | +A Spaceflake is structured like the following: |
| 12 | + |
| 13 | + |
| 14 | +## Spaceflake Network |
| 15 | +A Spaceflake Network is a very basic concept where you have multiple **independent nodes** that themselves consist of multiple workers. These workers are the ones that can generate a Spaceflake. |
| 16 | + |
| 17 | +Ideally a Spaceflake Network represents your entire application, or company. Each node represents a single server or application within the company, and each worker represents a single process which can generate a Spaceflake for a specific purpose. This way you can easily identify where a Spaceflake was generated by looking at its node ID and worker ID. |
| 18 | + |
| 19 | +In the end you are free to use them as you wish, just make sure you use these nodes and workers to be able to identify the Spaceflake. |
| 20 | + |
| 21 | +### Example Network |
| 22 | +An example network is structured like the following |
| 23 | + |
| 24 | +We can consider **Node 1** as being the API/backend of your application. The **Worker (ID: 1)** would be responsible for generating Spaceflakes for user IDs. The **Worker (ID: 2)** would be responsible for generating Spaceflakes for blog post IDs. |
| 25 | + |
| 26 | +The **Node 2** might be responsible for the logs of your components, and the log ID generated would be generated by the **Worker (ID: 1)** from that node. |
| 27 | + |
| 28 | +## Some Statistics |
| 29 | +* A Spaceflake network can hold up to **31 nodes** and **31 workers per node**. So you can have up to **961 workers in total** in a single network that will generate Spaceflakes. |
| 30 | +* A **single worker** can generate up to **4095 Spaceflakes per millisecond**. |
| 31 | +* A **single node** with **31 workers** can generate up to **126'945 Spaceflakes per millisecond**. |
| 32 | +* A **single network** with **31 nodes** and **31 workers per node** can generate up to **3'935'295 Spaceflakes per millisecond**. |
| 33 | + |
| 34 | +## Example |
| 35 | +A very basic example on using the library is by using the generator **without** nodes and worker objects, though this is not recommended and using nodes and workers is better. |
| 36 | +```go |
| 37 | +package main |
| 38 | + |
| 39 | +import ( |
| 40 | + "fmt" |
| 41 | + |
| 42 | + "github.com/kkrypt0nn/spaceflake" |
| 43 | +) |
| 44 | + |
| 45 | +func main() { |
| 46 | + node := spaceflake.NewNode(1) |
| 47 | + worker := node.NewWorker() |
| 48 | + sf, err := worker.GenerateSpaceflake() |
| 49 | + if err != nil { |
| 50 | + panic(err) |
| 51 | + } |
| 52 | + fmt.Println(sf.Decompose()) // map[id:<Spaceflake> nodeID:1 sequence:1 time:<timestamp> workerID:1] |
| 53 | +} |
| 54 | +``` |
| 55 | +Some other examples: |
| 56 | +- [Bulk generation](examples/bulk/bulk.go): Generate multiple Spaceflakes at once. |
| 57 | +- [Generation with nodes and workers](examples/node_workers/node_workers.go): Generate a Spaceflake by creating a node and worker object. |
| 58 | +- [Generation with settings](examples/generate/generate.go): Generate a Spaceflakes with specific settings *(Not recommended, consider using nodes and workers)* |
| 59 | + |
| 60 | +## Installation |
| 61 | + |
| 62 | +If you want to use this library for one of your projects, you can install it like any other Go library |
| 63 | + |
| 64 | +```shell |
| 65 | +go get github.com/kkrypt0nn/spaceflake |
| 66 | +``` |
| 67 | + |
| 68 | +## ⚠️ Disclaimers |
| 69 | +### Spaceflakes are Big Numbers |
| 70 | +> 📜 TL;DR: If you use Spaceflakes in an API, return them as a **string**, not a number. |
| 71 | +
|
| 72 | +Since Spaceflakes are big numbers, it is most likely that if you use them for an API that returns a JSON you will need to return the Spaceflake as a string, otherwise you will lose some precision and it will alter the value of, most likely, the sequence of the Spaceflake. Example: |
| 73 | +```json |
| 74 | +{ |
| 75 | + "id": 144328692659220480 // ID actually generated in Go: 144328692659220481 |
| 76 | +} |
| 77 | +``` |
| 78 | +The difference between the two numbers is not that big in the example above, though it plays a big role. The difference is not always the same, so you can't subtract. JavaScript, for example, sees no difference between both of these numbers: |
| 79 | +```js |
| 80 | +console.log(144328692659220480 == 144328692659220481) // true |
| 81 | +``` |
| 82 | + |
| 83 | +You can get the Spaceflake as a string and convert to a `uint64` data type, when needed, in your Go code using the following: |
| 84 | +```Go |
| 85 | +spaceflakeID := "144328692659220481" // Will be the value returned by the API |
| 86 | +id, _ := strconv.ParseUint(spaceflakeID, 10, 64) |
| 87 | +sequence := spaceflake.ParseSequence(id) |
| 88 | +``` |
| 89 | +### "Random" Sequence Based on Time |
| 90 | +> 📜 TL;DR: The sequence is not truly random, it is based on the time; and if you generate lots of Spaceflake in the same millisecond, there is a chance that two Spaceflakes will result to the same. Using **nodes and workers** is highly recommended. |
| 91 | +
|
| 92 | +When generating lots of Spaceflakes in a really short time and without using a worker, there is a chance that the same ID is generated twice. Consider making your program sleep for 1 millisecond or test around between the generations, example: |
| 93 | +```go |
| 94 | +func GenerateLotsOfSpaceflakes() { |
| 95 | + spaceflakes := map[uint64]*Spaceflake{} |
| 96 | + settings := NewGeneratorSettings() |
| 97 | + |
| 98 | + for i := 0; i < 1000; i++ { |
| 99 | + sf, err := Generate(settings) |
| 100 | + if err != nil { |
| 101 | + t.Error(err) |
| 102 | + } |
| 103 | + if spaceflakes[sf.ID()] != nil { |
| 104 | + panic(err) |
| 105 | + } |
| 106 | + spaceflakes[sf.ID()] = sf |
| 107 | + |
| 108 | + // When using random there is a chance that the sequence will be twice the same due to Go's speed, hence using a worker is better. We wait a millisecond to make sure it's different. |
| 109 | + time.Sleep(1 * time.Millisecond) |
| 110 | + } |
| 111 | +} |
| 112 | +``` |
| 113 | +In that case it is recommended to use the workers, as they do not use a random value as a sequence number, but an incrementing value. Another option would be to use the [bulk generator](examples/bulk/bulk.go) to create lots of **unique** Spaceflakes at once. |
| 114 | + |
| 115 | +As a last resort you can replace the sequence with a better random number generator using the following: |
| 116 | +```go |
| 117 | +settings.Sequence = ... // Replace with your number generator |
| 118 | +``` |
| 119 | + |
| 120 | +## License |
| 121 | + |
| 122 | +This library was made with 💜 by Krypton and is under the [GPL v3](LICENSE.md) license. |
0 commit comments