|
8 | 8 | <a href="[https://codecov.io/gh/field4s/field4s](https://codecov.io/gh/scafi/scafi3)"><img src="https://codecov.io/gh/scafi/scafi3/graph/badge.svg?token=5ZT5AEMNDF" alt="codecov"></a> |
9 | 9 | </p> |
10 | 10 |
|
11 | | -# Scafi 3 |
| 11 | +# ScaFi 3 |
12 | 12 |
|
13 | | -**scafi3** is a Scala 3 DSL for _Aggregate Programming_. |
| 13 | +**ScaFi 3** (Scala Fields 3) is a modern Scala 3 DSL and toolkit for **Aggregate Programming**, a paradigm for designing resilient and self-organizing distributed systems. |
14 | 14 |
|
15 | | -Roots its foundations on the concept of **computational fields**: |
16 | | -a _distributed data structure_ manipulated by _aggregate_ functions implementing the business logic for **large-scale distributed systems**. |
| 15 | +## 🌟 What is Aggregate Programming? |
| 16 | + |
| 17 | +Aggregate Programming enables you to program collective behaviors across networks of devices—from IoT sensors to robot swarms—by thinking in terms of **computational fields**: distributed data structures that span across the entire system. Instead of programming individual devices, you express global behaviors that automatically adapt to the network topology and evolve over time. |
| 18 | + |
| 19 | +## 🎯 Philosophy and Design |
| 20 | + |
| 21 | +ScaFi 3 embraces a **functional, composable, and type-safe** approach to distributed programming: |
| 22 | + |
| 23 | +### Computational Fields First |
| 24 | +At its core, ScaFi operates on **fields**: values mapped across space and time in a distributed network. A field might represent temperatures across sensors, distances from a source, or any aggregate value. Fields are first-class citizens that you can manipulate with aggregate operators. |
| 25 | + |
| 26 | +### Effect System Integration |
| 27 | +ScaFi 3 leverages Scala 3's advanced type system and **effect tracking** to provide compile-time safety guarantees: |
| 28 | + |
| 29 | +- **Safer Exceptions**: Using Scala 3's `throws` clauses, functions explicitly declare what exceptions they can throw, enabling better error handling at compile time |
| 30 | +- **Context Functions**: The framework uses `using`/`given` for implicit context passing, making the aggregate computation context available without boilerplate |
| 31 | +- **Type-Level Guarantees**: Strong typing ensures that field operations are correctly aligned and serialization is properly configured |
| 32 | + |
| 33 | +### Programming Style |
| 34 | + |
| 35 | +ScaFi programs are: |
| 36 | +- **Declarative**: Express *what* the system should compute, not *how* each device should behave |
| 37 | +- **Composable**: Build complex behaviors from simple, reusable building blocks |
| 38 | +- **Resilient**: Automatically handle device failures and network topology changes |
| 39 | +- **Pure**: Computation logic is separated from side effects and device-specific concerns |
| 40 | + |
| 41 | +## 📦 Installation |
| 42 | + |
| 43 | +### With SBT |
| 44 | + |
| 45 | +Add ScaFi 3 to your `build.sbt`: |
| 46 | + |
| 47 | +```scala |
| 48 | +libraryDependencies ++= Seq( |
| 49 | + "it.unibo.scafi" %%% "scafi3-core" % "1.0.5", |
| 50 | + // For distributed systems support |
| 51 | + "it.unibo.scafi" %%% "scafi3-distributed" % "1.0.5" |
| 52 | +) |
| 53 | +``` |
| 54 | + |
| 55 | +ScaFi 3 supports **JVM**, **JavaScript**, and **Native** platforms through Scala's cross-platform compilation: |
| 56 | + |
| 57 | +```scala |
| 58 | +// For a specific platform, use %% instead of %%% |
| 59 | +libraryDependencies += "it.unibo.scafi" %% "scafi3-core" % "1.0.5" |
| 60 | +``` |
| 61 | + |
| 62 | +### With Mill |
| 63 | + |
| 64 | +Add ScaFi 3 to your `build.sc`: |
| 65 | + |
| 66 | +```scala |
| 67 | +import mill._, scalalib._ |
| 68 | + |
| 69 | +object myproject extends ScalaModule { |
| 70 | + def scalaVersion = "3.7.3" |
| 71 | + |
| 72 | + def ivyDeps = Agg( |
| 73 | + ivy"it.unibo.scafi::scafi3-core:1.0.5", |
| 74 | + // For distributed systems support |
| 75 | + ivy"it.unibo.scafi::scafi3-distributed:1.0.5" |
| 76 | + ) |
| 77 | +} |
| 78 | +``` |
| 79 | + |
| 80 | +For cross-platform projects with Mill: |
| 81 | + |
| 82 | +```scala |
| 83 | +import mill._, scalalib._, scalajslib._, scalanativelib._ |
| 84 | + |
| 85 | +object myproject extends Cross[MyProjectModule](JVMPlatform, JSPlatform, NativePlatform) |
| 86 | + |
| 87 | +class MyProjectModule(val platform: Platform) extends CrossPlatformScalaModule { |
| 88 | + def scalaVersion = "3.7.3" |
| 89 | + |
| 90 | + def ivyDeps = Agg( |
| 91 | + ivy"it.unibo.scafi::scafi3-core::1.0.5", |
| 92 | + ) |
| 93 | +} |
| 94 | +``` |
| 95 | + |
| 96 | +## 🚀 Quick Start |
| 97 | + |
| 98 | +### Basic Field Operations |
| 99 | + |
| 100 | +The foundation of ScaFi is the **field calculus**, which provides three core primitives: |
| 101 | + |
| 102 | +```scala |
| 103 | +import it.unibo.scafi.language.xc.ExchangeLanguage |
| 104 | +import it.unibo.scafi.language.xc.calculus.ExchangeCalculus |
| 105 | +import it.unibo.scafi.libraries.FieldCalculusLibrary.* |
| 106 | +import it.unibo.scafi.libraries.CommonLibrary.localId |
| 107 | + |
| 108 | +// Define your aggregate program using context functions |
| 109 | +def myProgram(using context: ExchangeCalculus & ExchangeLanguage): Int = |
| 110 | + // neighborValues: share a value with neighbors and get their values back |
| 111 | + val neighborIds = neighborValues(localId) |
| 112 | + |
| 113 | + // evolve: maintain state across rounds (like a distributed variable) |
| 114 | + val roundCount = evolve(0)(count => count + 1) |
| 115 | + |
| 116 | + // share: compute values while sharing them with neighbors |
| 117 | + val consensus = share(0)(neighborValues => |
| 118 | + (neighborValues.toIterable.sum + roundCount) / (neighborValues.size + 1) |
| 119 | + ) |
| 120 | + |
| 121 | + consensus |
| 122 | +``` |
| 123 | + |
| 124 | +### Distance Gradient |
| 125 | + |
| 126 | +A classic aggregate programming example—computing distance from a source: |
| 127 | + |
| 128 | +```scala |
| 129 | +import it.unibo.scafi.libraries.GradientLibrary.* |
| 130 | +import it.unibo.scafi.libraries.CommonLibrary.* |
| 131 | +import it.unibo.scafi.message.Codables.given |
| 132 | + |
| 133 | +def gradient(using context: ExchangeCalculus & ExchangeLanguage): Double = |
| 134 | + // Compute distance from source using neighbor distance + edge cost |
| 135 | + distanceTo[Array[Byte], Double]( |
| 136 | + source = localId == 0, // Device 0 is the source |
| 137 | + distances = neighborValues(1.0) // Each hop costs 1.0 |
| 138 | + ) |
| 139 | +``` |
| 140 | + |
| 141 | +### Domain Splitting with Branching |
| 142 | + |
| 143 | +Control information flow with domain branching: |
| 144 | + |
| 145 | +```scala |
| 146 | +import it.unibo.scafi.libraries.BranchingLibrary.* |
| 147 | + |
| 148 | +def conditionalBehavior(using context: ExchangeCalculus & ExchangeLanguage): String = |
| 149 | + val distanceFromSource = gradient |
| 150 | + |
| 151 | + // Split the network into two independent computational domains |
| 152 | + branch(distanceFromSource < 5.0)( |
| 153 | + "Close to source: " + distanceFromSource |
| 154 | + )( |
| 155 | + "Far from source: " + distanceFromSource |
| 156 | + ) |
| 157 | +``` |
| 158 | + |
| 159 | +### Exchange Calculus: The Foundation |
| 160 | + |
| 161 | +Under the hood, ScaFi implements the **Exchange Calculus**, a more expressive variant of field calculus: |
| 162 | + |
| 163 | +```scala |
| 164 | +import it.unibo.scafi.libraries.ExchangeCalculusLibrary.* |
| 165 | +import it.unibo.scafi.language.xc.syntax.ReturnSending.returning |
| 166 | + |
| 167 | +def exchangeExample(using context: ExchangeCalculus & ExchangeLanguage): Int = |
| 168 | + // exchange: send and receive different values |
| 169 | + exchange(0) { receivedValues => |
| 170 | + val maxFromNeighbors = receivedValues.withoutSelf.max |
| 171 | + val myValue = localId + maxFromNeighbors |
| 172 | + |
| 173 | + // Return one value but send another to neighbors |
| 174 | + returning(myValue) send (myValue + 1) |
| 175 | + } |
| 176 | +``` |
| 177 | + |
| 178 | +### Complete Example: Self-Healing Gradient |
| 179 | + |
| 180 | +Here's a complete example showing ScaFi's resilience: |
| 181 | + |
| 182 | +```scala |
| 183 | +import it.unibo.scafi.language.xc.{ExchangeLanguage, ExchangeCalculus} |
| 184 | +import it.unibo.scafi.libraries.GradientLibrary.* |
| 185 | +import it.unibo.scafi.libraries.CommonLibrary.* |
| 186 | +import it.unibo.scafi.message.Codables.given |
| 187 | + |
| 188 | +// Define an aggregate program that computes a self-healing gradient |
| 189 | +def selfHealingGradient( |
| 190 | + isSource: Boolean, |
| 191 | + hopDistance: Double = 1.0 |
| 192 | +)(using context: ExchangeCalculus & ExchangeLanguage): Double = |
| 193 | + |
| 194 | + // Automatically computes and maintains shortest distance from source |
| 195 | + // Adapts to topology changes and device failures |
| 196 | + distanceTo[Array[Byte], Double](isSource, neighborValues(hopDistance)) |
| 197 | + |
| 198 | +// In practice, this would be executed by the ScaFi engine across devices |
| 199 | +// Each device runs the same program but operates on its local context |
| 200 | +``` |
| 201 | + |
| 202 | +## 🏗️ Key Concepts |
| 203 | + |
| 204 | +### Shared Data Types |
| 205 | +`SharedData[T]` represents a field value—a mapping from device IDs to values of type `T`. It includes: |
| 206 | +- The local value on the current device |
| 207 | +- Values received from aligned neighbors |
| 208 | +- Combinators for field manipulation (map, flatMap, etc.) |
| 209 | + |
| 210 | +### Alignment |
| 211 | +ScaFi automatically manages **alignment**: ensuring that values from different parts of a computation are correctly associated across devices. The alignment mechanism uses tokens derived from the program structure. |
| 212 | + |
| 213 | +### Context Functions |
| 214 | +The `using` syntax provides implicit access to the aggregate computation context, which includes: |
| 215 | +- Device ID and neighbor information |
| 216 | +- Message history for state evolution |
| 217 | +- Network communication primitives |
| 218 | + |
| 219 | +## 🔧 Advanced Features |
| 220 | + |
| 221 | +- **Multi-platform**: Run on JVM, JavaScript (browser/Node.js), and Native targets |
| 222 | +- **Type-safe serialization**: Automatic codecs with compile-time guarantees using the `CodableFromTo` type class |
| 223 | +- **Modular libraries**: Compose pre-built libraries for common patterns (gradients, broadcasting, leader election) |
| 224 | +- **Integration ready**: Distributed module for real network deployments with socket-based communication |
| 225 | + |
| 226 | +## 📚 Learn More |
| 227 | + |
| 228 | +- **Examples**: Check the test suite for comprehensive examples |
| 229 | +- **API Documentation**: [ScalaDoc](https://scafi.github.io/scafi3/) |
| 230 | +- **Research**: Based on the [Aggregate Computing](https://doi.org/10.1109/MC.2015.261) research |
| 231 | + |
| 232 | +## 📄 License |
| 233 | + |
| 234 | +ScaFi 3 is released under the [Apache 2.0 License](./LICENSE). |
0 commit comments