+225 -84

ARCHITECTURE.md

··· 120 120 repositories = [ 121 121 { name = "default", path = "/path/to/opam-repository" }, 122 122 ] 123 123 - compiler = "5.4.0" 123 123 + compiler = "5.2.0" 124 124 125 125 - # Optional: override default XDG cache location 126 126 - # vendor_cache = "/path/to/vendor-cache" 127 127 - 128 128 - [cargo] 129 129 - # Future 125 125 + # Optional: vendor cache location (default: $XDG_CACHE_HOME/unpac/vendor-cache) 126 126 + vendor_cache = "/path/to/vendor-cache" 130 127 131 128 [projects] 132 129 # Project existence only, no package lists ··· 136 133 137 134 ## Vendor Cache 138 135 139 139 - A bare git repository that caches fetched packages to avoid hitting upstream remotes. 136 136 + A bare git repository that caches fetched packages to avoid repeated network fetches across multiple unpac projects. 137 137 + 138 138 + ### Location Priority 139 139 + 140 140 + 1. CLI flag: `--cache /path/to/cache` 141 141 + 2. Environment variable: `UNPAC_VENDOR_CACHE=/path/to/cache` 142 142 + 3. Config file: `vendor_cache = "/path/to/cache"` in unpac.toml 143 143 + 4. Default: `$XDG_CACHE_HOME/unpac/vendor-cache` (or `~/.cache/unpac/vendor-cache`) 144 144 + 145 145 + ### Cache Structure 140 146 141 141 - **Default location**: `$XDG_CACHE_HOME/unpac/vendor-cache/` (via xdge) 147 147 + The vendor cache is a bare git repository with remotes named after their URLs: 142 148 143 143 - **Override**: Set `vendor_cache` in config or pass `--vendor-cache` on CLI. 149 149 + ``` 150 150 + /path/to/vendor-cache/ # Bare git repository 151 151 + ├── config 152 152 + ├── objects/ # Shared git objects 153 153 + ├── refs/ 154 154 + │ └── remotes/ 155 155 + │ ├── github.com/dbuenzli/astring/ 156 156 + │ │ └── master 157 157 + │ ├── github.com/dbuenzli/cmdliner/ 158 158 + │ │ └── master 159 159 + │ └── github.com/ocaml/dune/ 160 160 + │ └── main 161 161 + └── ... 162 162 + ``` 144 163 145 145 - The cache holds: 146 146 - - `opam/upstream/*` branches (pristine upstream) 147 147 - - `opam/vendor/*` branches (pre-built with vendor prefix) 164 164 + Each remote is named using URL-based naming: 165 165 + - `https://github.com/dbuenzli/astring.git` → remote `github.com/dbuenzli/astring` 166 166 + - `https://github.com/ocaml/dune.git` → remote `github.com/ocaml/dune` 167 167 + 168 168 + ### Cache Workflow 169 169 + 170 170 + When adding a package: 171 171 + 172 172 + 1. **Check cache**: Look for remote `github.com/owner/repo` in cache 173 173 + 2. **Fetch to cache**: If not present (or stale), fetch from upstream to cache 174 174 + 3. **Fetch to project**: Fetch from cache to project's `git/` bare repo 175 175 + 4. **Create branches**: Create `opam/upstream/<pkg>` etc. in project 176 176 + 177 177 + ``` 178 178 + Upstream Vendor Cache Project git/ 179 179 + (GitHub) (bare repo) (bare repo) 180 180 + │ │ │ 181 181 + │ fetch (if needed) │ │ 182 182 + ├──────────────────────────►│ │ 183 183 + │ │ fetch from cache │ 184 184 + │ ├───────────────────────────►│ 185 185 + │ │ │ 186 186 + ``` 187 187 + 188 188 + ### Cache Commands 189 189 + 190 190 + ```bash 191 191 + # Add package using cache 192 192 + unpac opam add --cache /path/to/cache https://github.com/foo/bar.git 193 193 + 194 194 + # Using environment variable 195 195 + export UNPAC_VENDOR_CACHE=/path/to/cache 196 196 + unpac opam add astring 197 197 + 198 198 + # Configure in unpac.toml 199 199 + vendor_cache = "/path/to/cache" 200 200 + ``` 201 201 + 202 202 + ### Cache Benefits 148 203 149 149 - Projects fetch from the cache as a git remote. 204 204 + - **Shared across projects**: Multiple unpac projects share the same cache 205 205 + - **Offline support**: Once cached, packages can be added without network 206 206 + - **Faster operations**: No redundant downloads for common packages 207 207 + - **Persistent**: Cache survives project deletion 150 208 151 209 ## Derivable State 152 210 ··· 181 239 1. Create orphan branch `project/myapp` with template: 182 240 - `dune-project` (lang dune 3.20) 183 241 - `dune` with `(vendored_dirs vendor)` 242 242 + - `vendor/opam/` directory 184 243 2. Create worktree `project/myapp/` 185 244 3. Add to `[projects]` in `main/unpac.toml` 186 245 187 187 - ### Add Packages 246 246 + ### Add Package (by URL) 247 247 + 248 248 + ```bash 249 249 + unpac opam add https://github.com/dbuenzli/astring.git 250 250 + ``` 251 251 + 252 252 + 1. Detect default branch from remote 253 253 + 2. If cache configured: fetch upstream → cache → project 254 254 + 3. If no cache: fetch upstream → project directly 255 255 + 4. Create `opam/upstream/astring` branch 256 256 + 5. Create `opam/vendor/astring` orphan branch (with vendor/opam/astring/ prefix) 257 257 + 6. Create `opam/patches/astring` branch (from vendor) 258 258 + 7. Cleanup temporary worktrees 259 259 + 260 260 + ### Add Package (by name) 261 261 + 262 262 + ```bash 263 263 + unpac opam repo add default /path/to/opam-repository 264 264 + unpac opam config compiler 5.2.0 265 265 + unpac opam add astring 266 266 + ``` 267 267 + 268 268 + 1. Look up package in configured opam repositories 269 269 + 2. Extract dev-repo URL from opam file 270 270 + 3. Proceed as with URL-based add 271 271 + 272 272 + ### Add with Dependency Solving 188 273 189 274 ```bash 190 190 - unpac opam add eio lwt --project=myapp 275 275 + unpac opam add eio --solve 191 276 ``` 192 277 193 193 - 1. Resolve dependencies (eio, lwt → full dependency tree) 194 194 - 2. For each package: 195 195 - a. Check vendor cache, fetch from upstream if missing 196 196 - b. Create `opam/upstream/<pkg>` branch 197 197 - c. Create `opam/vendor/<pkg>` orphan branch (with prefix) 198 198 - d. Create `opam/patches/<pkg>` branch (from vendor) 199 199 - 3. Merge all `opam/patches/*` into `project/myapp` 278 278 + 1. Solve dependencies using configured compiler version 279 279 + 2. For each package in solution: 280 280 + - Look up dev-repo URL 281 281 + - Add package (via cache if configured) 282 282 + 3. Report summary 283 283 + 284 284 + ### Merge into Project 285 285 + 286 286 + ```bash 287 287 + unpac opam merge astring myapp 288 288 + ``` 200 289 201 201 - Worktrees are created temporarily during operations and cleaned up after. 290 290 + 1. Merge `opam/patches/astring` into `project/myapp` (allow unrelated histories) 291 291 + 2. Package files appear at `project/myapp/vendor/opam/astring/` 202 292 203 293 ### Update Packages 204 294 205 295 ```bash 206 206 - unpac opam update eio --project=myapp 296 296 + unpac opam update astring 207 297 ``` 208 298 209 209 - 1. Fetch latest upstream into `opam/upstream/eio` 210 210 - 2. Update `opam/vendor/eio` with new content 211 211 - 3. Rebase `opam/patches/eio` onto new vendor 212 212 - - If conflicts: leave worktree for manual resolution 213 213 - 4. Merge updated patches into `project/myapp` 299 299 + 1. Fetch latest from upstream (via cache if configured) 300 300 + 2. Compare old vs new upstream SHA 301 301 + 3. If changed: 302 302 + - Update `opam/upstream/astring` branch 303 303 + - Update `opam/vendor/astring` with new content 304 304 + 4. Note: patches branch must be rebased separately 214 305 215 306 ### Edit Patches 216 307 ··· 226 317 unpac opam done astring 227 318 ``` 228 319 229 229 - 1. Remove worktree (keeps branch and commits) 320 320 + 1. Check for uncommitted changes 321 321 + 2. Remove worktree (keeps branch and commits) 322 322 + 323 323 + ### View Changes 230 324 231 325 ```bash 232 232 - # Then merge into project 233 233 - cd project/myapp 234 234 - git merge opam/patches/astring 326 326 + unpac opam diff astring 235 327 ``` 236 328 237 237 - ### Vendor Cache Operations 329 329 + Shows diff between `opam/vendor/astring` and `opam/patches/astring`. 330 330 + 331 331 + ### Remove Package 238 332 239 333 ```bash 240 240 - # Fetch packages into cache (without adding to any project) 241 241 - unpac cache fetch eio lwt --deps 334 334 + unpac opam remove astring 242 335 ``` 243 336 244 244 - 1. Resolve dependencies 245 245 - 2. Fetch each package into vendor cache 246 246 - 3. Create `opam/upstream/*` and `opam/vendor/*` branches in cache 337 337 + 1. Remove any existing worktrees 338 338 + 2. Delete `opam/upstream/astring`, `opam/vendor/astring`, `opam/patches/astring` branches 339 339 + 3. Remove remote `origin-astring` 247 340 248 341 ## Module Structure 249 342 ··· 254 347 ├── init.ml # Project initialization 255 348 ├── config.ml # TOML config parsing 256 349 ├── worktree.ml # Git worktree lifecycle management 350 350 + ├── vendor_cache.ml # Vendor cache (bare git repo) 257 351 ├── git.ml # Low-level git operations 258 352 ├── git_repo_lookup.ml # URL rewriting (erratique → github, etc.) 259 259 - ├── cache.ml # Vendor cache management 260 260 - │ 261 353 ├── backend.ml # Backend module signature 262 354 │ 263 263 - ├── opam/ # Opam backend 264 264 - │ ├── opam_backend.ml # Backend implementation 265 265 - │ ├── upstream.ml # opam/upstream/* management 266 266 - │ ├── vendor.ml # opam/vendor/* management 267 267 - │ ├── patches.ml # opam/patches/* management 268 268 - │ ├── solver.ml # Dependency resolution 269 269 - │ ├── source.ml # Source extraction 270 270 - │ ├── repo_index.ml # Repository indexing 271 271 - │ └── dev_repo.ml # Dev-repo normalization 272 272 - │ 273 273 - └── project.ml # Project branch operations 355 355 + └── opam/ # Opam backend 356 356 + ├── opam.ml # Main backend implementation 357 357 + ├── repo.ml # Repository package lookup 358 358 + └── solver.ml # Dependency resolution 274 359 275 360 bin/ 276 361 └── main.ml # CLI ··· 281 366 Manages worktree lifecycle within the unpac directory structure. 282 367 283 368 ```ocaml 284 284 - type root 369 369 + type root = Eio.Fs.dir_ty Eio.Path.t 285 370 (** The unpac project root (contains git/, main/, etc.) *) 286 371 287 372 type kind = ··· 291 376 | Opam_vendor of string 292 377 | Opam_patches of string 293 378 294 294 - val path : root -> kind -> path 379 379 + val path : root -> kind -> Eio.Fs.dir_ty Eio.Path.t 295 380 (** Filesystem path for a worktree kind. *) 296 381 297 382 val branch : kind -> string 298 383 (** Git branch name for a worktree kind. *) 299 384 300 300 - val ensure : proc_mgr -> root -> kind -> unit 385 385 + val ensure : proc_mgr:Git.proc_mgr -> root -> kind -> unit 301 386 (** Create worktree if it doesn't exist. *) 302 387 303 303 - val remove : proc_mgr -> root -> kind -> unit 388 388 + val ensure_orphan : proc_mgr:Git.proc_mgr -> root -> kind -> unit 389 389 + (** Create orphan worktree if it doesn't exist. *) 390 390 + 391 391 + val remove : proc_mgr:Git.proc_mgr -> root -> kind -> unit 304 392 (** Remove worktree (keeps branch). *) 305 393 306 306 - val with_temp : proc_mgr -> root -> kind -> (path -> 'a) -> 'a 394 394 + val with_temp : proc_mgr:Git.proc_mgr -> root -> kind -> (path -> 'a) -> 'a 307 395 (** Create worktree, run function, remove worktree. *) 308 396 ``` 309 397 398 398 + ### Key Module: `vendor_cache.ml` 399 399 + 400 400 + Manages the persistent vendor cache bare repository. 401 401 + 402 402 + ```ocaml 403 403 + type t = Eio.Fs.dir_ty Eio.Path.t 404 404 + (** Path to the cache bare repository *) 405 405 + 406 406 + val init : proc_mgr:Git.proc_mgr -> fs:Eio.Fs.dir_ty Eio.Path.t -> 407 407 + ?path:string -> unit -> t 408 408 + (** Initialize cache, creating bare repo if needed. *) 409 409 + 410 410 + val url_to_remote_name : string -> string 411 411 + (** Convert URL to remote name: "https://github.com/foo/bar.git" → "github.com/foo/bar" *) 412 412 + 413 413 + val fetch : proc_mgr:Git.proc_mgr -> t -> url:string -> string 414 414 + (** Fetch from URL into cache, returns remote name. *) 415 415 + 416 416 + val fetch_to_project : proc_mgr:Git.proc_mgr -> cache:t -> 417 417 + project_git:Eio.Fs.dir_ty Eio.Path.t -> 418 418 + url:string -> branch:string -> string 419 419 + (** Fetch via cache into project's git repo. Returns cache ref name. *) 420 420 + ``` 421 421 + 310 422 ### Key Module: `backend.ml` 311 423 312 424 Signature for package manager backends. 313 425 314 426 ```ocaml 315 315 - module type S = sig 316 316 - val name : string 317 317 - (** "opam", "cargo", etc. *) 427 427 + type package_info = { 428 428 + name : string; 429 429 + url : string; 430 430 + branch : string option; 431 431 + } 318 432 319 319 - val upstream_branch : string -> string 320 320 - val vendor_branch : string -> string 321 321 - val patches_branch : string -> string 322 322 - val vendor_path : string -> string 433 433 + type add_result = 434 434 + | Added of { name : string; sha : string } 435 435 + | Already_exists of string 436 436 + | Failed of { name : string; error : string } 323 437 324 324 - val add : 325 325 - proc_mgr -> root -> cache:path -> name:string -> url:string -> branch:string -> unit 438 438 + type update_result = 439 439 + | Updated of { name : string; old_sha : string; new_sha : string } 440 440 + | No_changes of string 441 441 + | Update_failed of { name : string; error : string } 326 442 327 327 - val update : 328 328 - proc_mgr -> root -> name:string -> unit 329 329 - end 443 443 + val merge_to_project : proc_mgr:Git.proc_mgr -> root:Worktree.root -> 444 444 + project:string -> patches_branch:string -> 445 445 + (unit, [`Conflict of string list]) result 330 446 ``` 331 447 332 448 ## CLI Commands ··· 341 457 unpac project list 342 458 List projects 343 459 344 344 - unpac project remove <name> 345 345 - Remove project branch and worktree 460 460 + unpac opam repo add <name> <path> 461 461 + Add opam repository for package lookups 462 462 + 463 463 + unpac opam repo list 464 464 + List configured repositories 465 465 + 466 466 + unpac opam repo remove <name> 467 467 + Remove opam repository 468 468 + 469 469 + unpac opam config compiler [version] 470 470 + Get or set OCaml compiler version 471 471 + 472 472 + unpac opam add <pkg|url> [--name NAME] [--branch BRANCH] [--solve] [--cache PATH] 473 473 + Add package by name or URL 474 474 + --solve: resolve dependencies and add all 475 475 + --cache: use vendor cache at PATH 346 476 347 347 - unpac opam add <pkg...> --project=<name> [--deps] 348 348 - Add packages to project (--deps is default) 477 477 + unpac opam list 478 478 + List vendored packages 349 479 350 350 - unpac opam update <pkg...> --project=<name> 351 351 - Update packages from upstream 480 480 + unpac opam info <pkg> 481 481 + Show package information 352 482 353 353 - unpac opam remove <pkg...> --project=<name> 354 354 - Remove packages from project 483 483 + unpac opam diff <pkg> 484 484 + Show local changes (patches vs vendor) 355 485 356 486 unpac opam edit <pkg> 357 487 Create patches worktree for editing 358 488 359 489 unpac opam done <pkg> 360 360 - Remove patches worktree 490 490 + Close patches worktree 361 491 362 362 - unpac opam status [--project=<name>] 363 363 - Show package status 492 492 + unpac opam update <pkg> 493 493 + Update package from upstream 364 494 365 365 - unpac cache fetch <pkg...> [--deps] 366 366 - Fetch packages into vendor cache 495 495 + unpac opam merge <pkg> <project> 496 496 + Merge package into project 367 497 368 368 - unpac cache status 369 369 - Show cache status 498 498 + unpac opam remove <pkg> 499 499 + Remove vendored package 370 500 ``` 501 501 + 502 502 + ## URL Rewriting 503 503 + 504 504 + The `git_repo_lookup.ml` module rewrites URLs for known mirrors: 505 505 + 506 506 + | Original | Rewritten | 507 507 + |----------|-----------| 508 508 + | `https://erratique.ch/repos/fmt` | `https://github.com/dbuenzli/fmt` | 509 509 + | `git+https://...` | `https://...` (strip git+ prefix) | 510 510 + 511 511 + This ensures we fetch from reliable mirrors when available. 371 512 372 513 ## Future: Cargo Backend 373 514

+324

CLI.md

··· 1 1 + # Unpac CLI Workflow 2 2 + 3 3 + ## Overview 4 4 + 5 5 + Unpac is a vendoring tool that lets you maintain local patches to dependencies while tracking upstream changes. This document describes the ideal CLI workflow. 6 6 + 7 7 + ## Quick Start 8 8 + 9 9 + ```bash 10 10 + # Initialize a new unpac workspace 11 11 + unpac init myworkspace 12 12 + cd myworkspace 13 13 + 14 14 + # Configure an opam repository 15 15 + unpac opam repo add opam-repository /path/to/opam-repository 16 16 + 17 17 + # Create a project 18 18 + unpac project new myapp 19 19 + 20 20 + # Add a dependency (looks up dev-repo in opam repository) 21 21 + unpac opam add cmdliner 22 22 + 23 23 + # Edit the vendored package (opens patches worktree) 24 24 + unpac opam edit cmdliner 25 25 + # ... make changes, e.g., port to dune ... 26 26 + cd opam/patches/cmdliner 27 27 + git add -A && git commit -m "Port cmdliner to dune" 28 28 + cd ../../.. 29 29 + 30 30 + # Add the patched package to your project 31 31 + unpac opam merge cmdliner myapp 32 32 + 33 33 + # Build your project 34 34 + cd project/myapp 35 35 + dune build 36 36 + ``` 37 37 + 38 38 + ## Commands 39 39 + 40 40 + ### Initialization 41 41 + 42 42 + #### `unpac init <path>` 43 43 + 44 44 + Initialize a new unpac workspace at the given path. 45 45 + 46 46 + ```bash 47 47 + unpac init myworkspace 48 48 + ``` 49 49 + 50 50 + Creates: 51 51 + - `git/` - bare git repository (shared object store) 52 52 + - `main/` - metadata branch with `unpac.toml` 53 53 + 54 54 + ### Project Management 55 55 + 56 56 + #### `unpac project new <name>` 57 57 + 58 58 + Create a new project branch. 59 59 + 60 60 + ```bash 61 61 + unpac project new myapp 62 62 + ``` 63 63 + 64 64 + Creates an orphan branch `project/<name>` with: 65 65 + - `dune-project` with dune lang 3.20 66 66 + - `dune` with `(vendored_dirs vendor)` 67 67 + - `vendor/opam/` directory structure 68 68 + 69 69 + #### `unpac project list` 70 70 + 71 71 + List all projects in the workspace. 72 72 + 73 73 + ```bash 74 74 + unpac project list 75 75 + ``` 76 76 + 77 77 + ### Opam Repository Management 78 78 + 79 79 + #### `unpac opam repo add <name> <path-or-url>` 80 80 + 81 81 + Add an opam repository for package lookups. 82 82 + 83 83 + ```bash 84 84 + # Local repository 85 85 + unpac opam repo add opam-repository /workspace/opam/opam-repository 86 86 + 87 87 + # Remote repository (planned) 88 88 + unpac opam repo add opam-repository https://github.com/ocaml/opam-repository 89 89 + ``` 90 90 + 91 91 + #### `unpac opam repo list` 92 92 + 93 93 + List configured opam repositories. 94 94 + 95 95 + ```bash 96 96 + unpac opam repo list 97 97 + ``` 98 98 + 99 99 + #### `unpac opam repo remove <name>` 100 100 + 101 101 + Remove an opam repository. 102 102 + 103 103 + ```bash 104 104 + unpac opam repo remove opam-repository 105 105 + ``` 106 106 + 107 107 + ### Package Vendoring 108 108 + 109 109 + #### `unpac opam add <package-or-url> [--name <name>] [--version <version>]` 110 110 + 111 111 + Vendor a package. Can specify either: 112 112 + - A package name (looks up dev-repo in configured repositories) 113 113 + - A git URL directly 114 114 + 115 115 + ```bash 116 116 + # By package name (recommended) 117 117 + unpac opam add cmdliner 118 118 + unpac opam add cmdliner --version 1.3.0 119 119 + 120 120 + # By URL (for packages not in a repository) 121 121 + unpac opam add https://github.com/dbuenzli/cmdliner.git --name cmdliner 122 122 + ``` 123 123 + 124 124 + This creates three branches: 125 125 + - `opam/upstream/<pkg>` - pristine upstream code 126 126 + - `opam/vendor/<pkg>` - code with `vendor/opam/<pkg>/` path prefix 127 127 + - `opam/patches/<pkg>` - your local modifications (initially same as vendor) 128 128 + 129 129 + #### `unpac opam list` 130 130 + 131 131 + List all vendored packages. 132 132 + 133 133 + ```bash 134 134 + unpac opam list 135 135 + ``` 136 136 + 137 137 + #### `unpac opam edit <package>` 138 138 + 139 139 + Open the patches worktree for editing a package. 140 140 + 141 141 + ```bash 142 142 + unpac opam edit cmdliner 143 143 + ``` 144 144 + 145 145 + Creates/checks out `opam/patches/cmdliner/` worktree. Make your changes there, then commit: 146 146 + 147 147 + ```bash 148 148 + cd opam/patches/cmdliner 149 149 + # ... edit files ... 150 150 + git add -A 151 151 + git commit -m "Port to dune build system" 152 152 + ``` 153 153 + 154 154 + #### `unpac opam done <package>` 155 155 + 156 156 + Close the patches worktree (cleanup after editing). 157 157 + 158 158 + ```bash 159 159 + unpac opam done cmdliner 160 160 + ``` 161 161 + 162 162 + #### `unpac opam update <package>` 163 163 + 164 164 + Update a package from upstream. 165 165 + 166 166 + ```bash 167 167 + unpac opam update cmdliner 168 168 + ``` 169 169 + 170 170 + This: 171 171 + 1. Fetches latest from upstream 172 172 + 2. Updates `opam/upstream/<pkg>` and `opam/vendor/<pkg>` 173 173 + 3. Prints instructions for rebasing patches if needed 174 174 + 175 175 + #### `unpac opam rebase <package>` 176 176 + 177 177 + Rebase your patches onto the updated vendor branch. 178 178 + 179 179 + ```bash 180 180 + unpac opam rebase cmdliner 181 181 + ``` 182 182 + 183 183 + Opens the patches worktree for conflict resolution if needed. 184 184 + 185 185 + #### `unpac opam merge <package> <project>` 186 186 + 187 187 + Merge a vendored package into a project. 188 188 + 189 189 + ```bash 190 190 + unpac opam merge cmdliner myapp 191 191 + ``` 192 192 + 193 193 + Merges `opam/patches/<pkg>` into `project/<name>`, placing files under `vendor/opam/<pkg>/`. 194 194 + 195 195 + #### `unpac opam remove <package>` 196 196 + 197 197 + Remove a vendored package. 198 198 + 199 199 + ```bash 200 200 + unpac opam remove cmdliner 201 201 + ``` 202 202 + 203 203 + ### Package Information 204 204 + 205 205 + #### `unpac opam info <package>` 206 206 + 207 207 + Show information about a vendored package. 208 208 + 209 209 + ```bash 210 210 + unpac opam info cmdliner 211 211 + ``` 212 212 + 213 213 + Shows: 214 214 + - Upstream URL 215 215 + - Current upstream SHA 216 216 + - Current patches SHA 217 217 + - Number of local commits 218 218 + - Projects using this package 219 219 + 220 220 + #### `unpac opam diff <package>` 221 221 + 222 222 + Show the diff between vendor and patches (your local changes). 223 223 + 224 224 + ```bash 225 225 + unpac opam diff cmdliner 226 226 + ``` 227 227 + 228 228 + ## Workflow Examples 229 229 + 230 230 + ### Porting a Package to Dune 231 231 + 232 232 + ```bash 233 233 + # Setup 234 234 + unpac init myworkspace && cd myworkspace 235 235 + unpac opam repo add opam-repository /workspace/opam/opam-repository 236 236 + unpac project new myapp 237 237 + 238 238 + # Vendor and patch 239 239 + unpac opam add cmdliner 240 240 + unpac opam edit cmdliner 241 241 + 242 242 + cd opam/patches/cmdliner 243 243 + # Add dune files, remove _tags, etc. 244 244 + git add -A 245 245 + git commit -m "Port cmdliner to dune" 246 246 + cd ../../.. 247 247 + 248 248 + unpac opam done cmdliner 249 249 + 250 250 + # Use in project 251 251 + unpac opam merge cmdliner myapp 252 252 + cd project/myapp 253 253 + dune build 254 254 + ``` 255 255 + 256 256 + ### Updating a Patched Package 257 257 + 258 258 + ```bash 259 259 + # Update from upstream 260 260 + unpac opam update cmdliner 261 261 + 262 262 + # Rebase your patches 263 263 + unpac opam rebase cmdliner 264 264 + 265 265 + # If conflicts, resolve them: 266 266 + cd opam/patches/cmdliner 267 267 + # ... resolve conflicts ... 268 268 + git add -A 269 269 + git rebase --continue 270 270 + cd ../../.. 271 271 + 272 272 + unpac opam done cmdliner 273 273 + 274 274 + # Re-merge into project 275 275 + unpac opam merge cmdliner myapp 276 276 + ``` 277 277 + 278 278 + ### Adding Multiple Dependencies 279 279 + 280 280 + ```bash 281 281 + unpac opam add fmt 282 282 + unpac opam add logs 283 283 + unpac opam add cmdliner 284 284 + 285 285 + # Merge all into project 286 286 + unpac opam merge fmt myapp 287 287 + unpac opam merge logs myapp 288 288 + unpac opam merge cmdliner myapp 289 289 + ``` 290 290 + 291 291 + ## Directory Structure 292 292 + 293 293 + After setup, your workspace looks like: 294 294 + 295 295 + ``` 296 296 + myworkspace/ 297 297 + ├── git/ # Bare git repo (shared objects) 298 298 + ├── main/ # Metadata worktree 299 299 + │ └── unpac.toml # Configuration 300 300 + ├── project/ 301 301 + │ └── myapp/ # Project worktree 302 302 + │ ├── dune-project 303 303 + │ ├── dune 304 304 + │ └── vendor/ 305 305 + │ └── opam/ 306 306 + │ ├── cmdliner/ # Merged vendor code 307 307 + │ └── fmt/ 308 308 + └── opam/ # Package worktrees (on-demand) 309 309 + └── patches/ 310 310 + └── cmdliner/ # When editing 311 311 + ``` 312 312 + 313 313 + ## Configuration (unpac.toml) 314 314 + 315 315 + ```toml 316 316 + [opam] 317 317 + repositories = [ 318 318 + { name = "opam-repository", path = "/workspace/opam/opam-repository" } 319 319 + ] 320 320 + # compiler = "5.4.0" # Optional: pin compiler version 321 321 + 322 322 + [projects] 323 323 + myapp = {} 324 324 + ```

-18

TODO.md

··· 1 1 - # TODO 2 2 - 3 3 - ## Vendor Cache Git Repository 4 4 - 5 5 - Add a persistent vendor cache as a bare git repository in the XDG cache directory 6 6 - (`$XDG_CACHE_HOME/unpac/vendor-cache` or `~/.cache/unpac/vendor-cache`). 7 7 - 8 8 - This cache would: 9 9 - - Store fetched upstream repositories as branches (e.g., `github.com/dbuenzli/astring/master`) 10 10 - - Persist across multiple unpac project runs 11 11 - - Save network fetches when the same package is used in multiple projects 12 12 - - Allow offline operations when packages are already cached 13 13 - 14 14 - Implementation notes: 15 15 - - Initialize bare git repo on first use 16 16 - - Add remotes and fetch before cloning into project's upstream branches 17 17 - - Use `git fetch --all` to update cache 18 18 - - Store branch names using URL-based naming (e.g., `github.com/owner/repo/branch`)

+601 -39

bin/main.ml

··· 22 22 | Some root -> 23 23 f ~env ~fs ~proc_mgr ~root 24 24 25 25 + (* Helper to get config path *) 26 26 + let config_path root = 27 27 + let main_path = Unpac.Worktree.path root Unpac.Worktree.Main in 28 28 + Eio.Path.(main_path / "unpac.toml") |> snd 29 29 + 30 30 + (* Helper to load config *) 31 31 + let load_config root = 32 32 + let path = config_path root in 33 33 + match Unpac.Config.load path with 34 34 + | Ok cfg -> cfg 35 35 + | Error _ -> Unpac.Config.empty 36 36 + 37 37 + (* Helper to save config and commit *) 38 38 + let save_config ~proc_mgr root config msg = 39 39 + let path = config_path root in 40 40 + Unpac.Config.save_exn path config; 41 41 + let main_wt = Unpac.Worktree.path root Unpac.Worktree.Main in 42 42 + Unpac.Git.run_exn ~proc_mgr ~cwd:main_wt ["add"; "unpac.toml"] |> ignore; 43 43 + Unpac.Git.run_exn ~proc_mgr ~cwd:main_wt ["commit"; "-m"; msg] |> ignore 44 44 + 45 45 + (* Check if string looks like a URL or path (vs a package name) *) 46 46 + let is_url_or_path s = 47 47 + String.starts_with ~prefix:"http://" s || 48 48 + String.starts_with ~prefix:"https://" s || 49 49 + String.starts_with ~prefix:"git@" s || 50 50 + String.starts_with ~prefix:"git://" s || 51 51 + String.starts_with ~prefix:"ssh://" s || 52 52 + String.starts_with ~prefix:"file://" s || 53 53 + String.starts_with ~prefix:"/" s || (* Absolute path *) 54 54 + String.starts_with ~prefix:"./" s || (* Relative path *) 55 55 + String.starts_with ~prefix:"../" s || (* Relative path *) 56 56 + String.contains s ':' (* URL with scheme *) 57 57 + 58 58 + (* Helper to resolve vendor cache *) 59 59 + let resolve_cache ~proc_mgr ~fs ~config ~cli_cache = 60 60 + match Unpac.Config.resolve_vendor_cache ?cli_override:cli_cache config with 61 61 + | None -> None 62 62 + | Some path -> 63 63 + Some (Unpac.Vendor_cache.init ~proc_mgr ~fs ~path ()) 64 64 + 25 65 (* Init command *) 26 66 let init_cmd = 27 67 let doc = "Initialize a new unpac project." in ··· 34 74 let fs = Eio.Stdenv.fs env in 35 75 let proc_mgr = Eio.Stdenv.process_mgr env in 36 76 let _root = Unpac.Init.init ~proc_mgr ~fs path in 37 37 - Format.printf "Initialized unpac project at %s@." path 77 77 + Format.printf "Initialized unpac project at %s@." path; 78 78 + Format.printf "@.Next steps:@."; 79 79 + Format.printf " cd %s@." path; 80 80 + Format.printf " unpac opam repo add <name> <path> # configure opam repository@."; 81 81 + Format.printf " unpac project new <name> # create a project@." 38 82 in 39 83 let info = Cmd.info "init" ~doc in 40 84 Cmd.v info Term.(const run $ logging_term $ path_arg) ··· 49 93 let run () name = 50 94 with_root @@ fun ~env:_ ~fs:_ ~proc_mgr ~root -> 51 95 let _path = Unpac.Init.create_project ~proc_mgr root name in 52 52 - Format.printf "Created project %s@." name 96 96 + Format.printf "Created project %s@." name; 97 97 + Format.printf "@.Next steps:@."; 98 98 + Format.printf " unpac opam add <package> # vendor a package@."; 99 99 + Format.printf " unpac opam merge <package> %s # merge package into project@." name 53 100 in 54 101 let info = Cmd.info "new" ~doc in 55 102 Cmd.v info Term.(const run $ logging_term $ name_arg) ··· 71 118 let info = Cmd.info "project" ~doc in 72 119 Cmd.group info [project_new_cmd; project_list_cmd] 73 120 74 74 - (* Opam add command *) 121 121 + (* Opam repo add command *) 122 122 + let opam_repo_add_cmd = 123 123 + let doc = "Add an opam repository for package lookups." in 124 124 + let name_arg = 125 125 + let doc = "Name for the repository." in 126 126 + Arg.(required & pos 0 (some string) None & info [] ~docv:"NAME" ~doc) 127 127 + in 128 128 + let path_arg = 129 129 + let doc = "Path to the repository (local directory)." in 130 130 + Arg.(required & pos 1 (some string) None & info [] ~docv:"PATH" ~doc) 131 131 + in 132 132 + let run () name path = 133 133 + with_root @@ fun ~env:_ ~fs:_ ~proc_mgr ~root -> 134 134 + let config = load_config root in 135 135 + (* Check if already exists *) 136 136 + if Unpac.Config.find_repo config name <> None then begin 137 137 + Format.eprintf "Repository '%s' already exists@." name; 138 138 + exit 1 139 139 + end; 140 140 + (* Resolve to absolute path *) 141 141 + let abs_path = 142 142 + if Filename.is_relative path then 143 143 + Filename.concat (Sys.getcwd ()) path 144 144 + else path 145 145 + in 146 146 + (* Check path exists *) 147 147 + if not (Sys.file_exists abs_path && Sys.is_directory abs_path) then begin 148 148 + Format.eprintf "Error: '%s' is not a valid directory@." abs_path; 149 149 + exit 1 150 150 + end; 151 151 + let repo : Unpac.Config.repo_config = { 152 152 + repo_name = name; 153 153 + source = Local abs_path; 154 154 + } in 155 155 + let config' = Unpac.Config.add_repo config repo in 156 156 + save_config ~proc_mgr root config' (Printf.sprintf "Add repository %s" name); 157 157 + Format.printf "Added repository %s at %s@." name abs_path; 158 158 + Format.printf "@.Next: unpac opam add <package> # vendor a package by name@." 159 159 + in 160 160 + let info = Cmd.info "add" ~doc in 161 161 + Cmd.v info Term.(const run $ logging_term $ name_arg $ path_arg) 162 162 + 163 163 + (* Opam repo list command *) 164 164 + let opam_repo_list_cmd = 165 165 + let doc = "List configured opam repositories." in 166 166 + let run () = 167 167 + with_root @@ fun ~env:_ ~fs:_ ~proc_mgr:_ ~root -> 168 168 + let config = load_config root in 169 169 + if config.opam.repositories = [] then begin 170 170 + Format.printf "No repositories configured@."; 171 171 + Format.printf "@.Hint: unpac opam repo add <name> <path>@." 172 172 + end else 173 173 + List.iter (fun (r : Unpac.Config.repo_config) -> 174 174 + let path = match r.source with 175 175 + | Local p -> p 176 176 + | Remote u -> u 177 177 + in 178 178 + Format.printf "%s: %s@." r.repo_name path 179 179 + ) config.opam.repositories 180 180 + in 181 181 + let info = Cmd.info "list" ~doc in 182 182 + Cmd.v info Term.(const run $ logging_term) 183 183 + 184 184 + (* Opam repo remove command *) 185 185 + let opam_repo_remove_cmd = 186 186 + let doc = "Remove an opam repository." in 187 187 + let name_arg = 188 188 + let doc = "Name of the repository to remove." in 189 189 + Arg.(required & pos 0 (some string) None & info [] ~docv:"NAME" ~doc) 190 190 + in 191 191 + let run () name = 192 192 + with_root @@ fun ~env:_ ~fs:_ ~proc_mgr ~root -> 193 193 + let config = load_config root in 194 194 + if Unpac.Config.find_repo config name = None then begin 195 195 + Format.eprintf "Repository '%s' not found@." name; 196 196 + exit 1 197 197 + end; 198 198 + let config' = Unpac.Config.remove_repo config name in 199 199 + save_config ~proc_mgr root config' (Printf.sprintf "Remove repository %s" name); 200 200 + Format.printf "Removed repository %s@." name 201 201 + in 202 202 + let info = Cmd.info "remove" ~doc in 203 203 + Cmd.v info Term.(const run $ logging_term $ name_arg) 204 204 + 205 205 + (* Opam repo command group *) 206 206 + let opam_repo_cmd = 207 207 + let doc = "Manage opam repositories." in 208 208 + let info = Cmd.info "repo" ~doc in 209 209 + Cmd.group info [opam_repo_add_cmd; opam_repo_list_cmd; opam_repo_remove_cmd] 210 210 + 211 211 + (* Opam config compiler command *) 212 212 + let opam_config_compiler_cmd = 213 213 + let doc = "Set or show the OCaml compiler version for dependency solving." in 214 214 + let version_arg = 215 215 + let doc = "OCaml version to use (e.g., 5.2.0)." in 216 216 + Arg.(value & pos 0 (some string) None & info [] ~docv:"VERSION" ~doc) 217 217 + in 218 218 + let run () version_opt = 219 219 + with_root @@ fun ~env:_ ~fs:_ ~proc_mgr ~root -> 220 220 + let config = load_config root in 221 221 + match version_opt with 222 222 + | None -> 223 223 + (* Show current compiler *) 224 224 + (match Unpac.Config.get_compiler config with 225 225 + | Some v -> Format.printf "Compiler: %s@." v 226 226 + | None -> Format.printf "No compiler configured@.@.Hint: unpac opam config compiler 5.2.0@.") 227 227 + | Some version -> 228 228 + (* Set compiler *) 229 229 + let config' = Unpac.Config.set_compiler config version in 230 230 + save_config ~proc_mgr root config' (Printf.sprintf "Set compiler to %s" version); 231 231 + Format.printf "Compiler set to %s@." version 232 232 + in 233 233 + let info = Cmd.info "compiler" ~doc in 234 234 + Cmd.v info Term.(const run $ logging_term $ version_arg) 235 235 + 236 236 + (* Opam config command group *) 237 237 + let opam_config_cmd = 238 238 + let doc = "Configure opam settings." in 239 239 + let info = Cmd.info "config" ~doc in 240 240 + Cmd.group info [opam_config_compiler_cmd] 241 241 + 242 242 + (* Helper to add a single package by name *) 243 243 + let add_single_package ~proc_mgr ~root ?cache ~config ~name ~version_opt ~branch_opt () = 244 244 + let repos = config.Unpac.Config.opam.repositories in 245 245 + match Unpac_opam.Repo.find_package ~repos ~name ?version:version_opt () with 246 246 + | None -> 247 247 + Format.eprintf "Package '%s' not found in configured repositories@." name; 248 248 + `Failed 249 249 + | Some result -> 250 250 + match result.metadata.dev_repo with 251 251 + | None -> 252 252 + Format.eprintf "Package '%s' has no dev-repo field, skipping@." name; 253 253 + `Skipped 254 254 + | Some dev_repo -> 255 255 + (* Strip git+ prefix if present (opam dev-repo format) *) 256 256 + let url = if String.starts_with ~prefix:"git+" dev_repo then 257 257 + String.sub dev_repo 4 (String.length dev_repo - 4) 258 258 + else dev_repo in 259 259 + let info : Unpac.Backend.package_info = { 260 260 + name; 261 261 + url; 262 262 + branch = branch_opt; 263 263 + } in 264 264 + match Unpac_opam.Opam.add_package ~proc_mgr ~root ?cache info with 265 265 + | Unpac.Backend.Added { name = pkg_name; sha } -> 266 266 + Format.printf "Added %s (%s)@." pkg_name (String.sub sha 0 7); 267 267 + `Added 268 268 + | Unpac.Backend.Already_exists pkg_name -> 269 269 + Format.printf "Package %s already vendored@." pkg_name; 270 270 + `Exists 271 271 + | Unpac.Backend.Failed { name = pkg_name; error } -> 272 272 + Format.eprintf "Error adding %s: %s@." pkg_name error; 273 273 + `Failed 274 274 + 275 275 + (* Opam add command - enhanced to support package names and dependency solving *) 75 276 let opam_add_cmd = 76 76 - let doc = "Vendor an opam package from a git URL." in 77 77 - let url_arg = 78 78 - let doc = "Git URL of the package repository." in 79 79 - Arg.(required & pos 0 (some string) None & info [] ~docv:"URL" ~doc) 277 277 + let doc = "Vendor an opam package (by name or git URL)." in 278 278 + let pkg_arg = 279 279 + let doc = "Package name or git URL." in 280 280 + Arg.(required & pos 0 (some string) None & info [] ~docv:"PACKAGE" ~doc) 80 281 in 81 282 let name_arg = 82 82 - let doc = "Package name (defaults to repository name)." in 283 283 + let doc = "Override package name." in 83 284 Arg.(value & opt (some string) None & info ["n"; "name"] ~docv:"NAME" ~doc) 84 285 in 286 286 + let version_arg = 287 287 + let doc = "Package version (when adding by name)." in 288 288 + Arg.(value & opt (some string) None & info ["V"; "pkg-version"] ~docv:"VERSION" ~doc) 289 289 + in 85 290 let branch_arg = 86 291 let doc = "Git branch to vendor (defaults to remote default)." in 87 292 Arg.(value & opt (some string) None & info ["b"; "branch"] ~docv:"BRANCH" ~doc) 88 293 in 89 89 - let run () url name_opt branch_opt = 90 90 - with_root @@ fun ~env:_ ~fs:_ ~proc_mgr ~root -> 91 91 - let name = match name_opt with 92 92 - | Some n -> n 93 93 - | None -> 94 94 - (* Extract name from URL *) 95 95 - let base = Filename.basename url in 96 96 - if String.ends_with ~suffix:".git" base then 97 97 - String.sub base 0 (String.length base - 4) 98 98 - else base 99 99 - in 100 100 - let info : Unpac.Backend.package_info = { 101 101 - name; 102 102 - url; 103 103 - branch = branch_opt; 104 104 - } in 105 105 - match Unpac_opam.Opam.add_package ~proc_mgr ~root info with 106 106 - | Unpac.Backend.Added { name; sha } -> 107 107 - Format.printf "Added %s (%s)@." name (String.sub sha 0 7) 108 108 - | Unpac.Backend.Already_exists name -> 109 109 - Format.printf "Package %s already vendored@." name 110 110 - | Unpac.Backend.Failed { name; error } -> 111 111 - Format.eprintf "Error adding %s: %s@." name error; 294 294 + let solve_arg = 295 295 + let doc = "Solve dependencies and vendor all required packages." in 296 296 + Arg.(value & flag & info ["solve"] ~doc) 297 297 + in 298 298 + let cache_arg = 299 299 + let doc = "Path to vendor cache (overrides config and UNPAC_VENDOR_CACHE env var)." in 300 300 + Arg.(value & opt (some string) None & info ["cache"] ~docv:"PATH" ~doc) 301 301 + in 302 302 + let run () pkg name_opt version_opt branch_opt solve cli_cache = 303 303 + with_root @@ fun ~env:_ ~fs ~proc_mgr ~root -> 304 304 + let config = load_config root in 305 305 + let cache = resolve_cache ~proc_mgr ~fs ~config ~cli_cache in 306 306 + 307 307 + if solve then begin 308 308 + (* Solve dependencies and add all packages *) 309 309 + let repos = config.opam.repositories in 310 310 + if repos = [] then begin 311 311 + Format.eprintf "No repositories configured. Add one with: unpac opam repo add <name> <path>@."; 112 312 exit 1 313 313 + end; 314 314 + let ocaml_version = match Unpac.Config.get_compiler config with 315 315 + | Some v -> v 316 316 + | None -> 317 317 + Format.eprintf "No compiler version configured.@."; 318 318 + Format.eprintf "Set one with: unpac opam config compiler 5.2.0@."; 319 319 + exit 1 320 320 + in 321 321 + (* Get repo paths *) 322 322 + let repo_paths = List.map (fun (r : Unpac.Config.repo_config) -> 323 323 + match r.source with 324 324 + | Unpac.Config.Local p -> p 325 325 + | Unpac.Config.Remote u -> u (* TODO: handle remote repos *) 326 326 + ) repos in 327 327 + Format.printf "Solving dependencies for %s...@." pkg; 328 328 + match Unpac_opam.Solver.solve ~repos:repo_paths ~ocaml_version ~packages:[pkg] with 329 329 + | Error msg -> 330 330 + Format.eprintf "Dependency solving failed:@.%s@." msg; 331 331 + exit 1 332 332 + | Ok result -> 333 333 + let pkgs = result.packages in 334 334 + Format.printf "Solution found: %d packages@." (List.length pkgs); 335 335 + List.iter (fun p -> 336 336 + Format.printf " %s.%s@." 337 337 + (OpamPackage.Name.to_string (OpamPackage.name p)) 338 338 + (OpamPackage.Version.to_string (OpamPackage.version p)) 339 339 + ) pkgs; 340 340 + Format.printf "@.Vendoring packages...@."; 341 341 + let added = ref 0 in 342 342 + let failed = ref 0 in 343 343 + List.iter (fun p -> 344 344 + let name = OpamPackage.Name.to_string (OpamPackage.name p) in 345 345 + match add_single_package ~proc_mgr ~root ?cache ~config ~name ~version_opt:None ~branch_opt () with 346 346 + | `Added -> incr added 347 347 + | `Exists -> () 348 348 + | `Skipped -> () 349 349 + | `Failed -> incr failed 350 350 + ) pkgs; 351 351 + Format.printf "@.Done: %d added, %d failed@." !added !failed; 352 352 + if !failed > 0 then exit 1 353 353 + end else begin 354 354 + (* Single package mode *) 355 355 + let url, name = 356 356 + if is_url_or_path pkg then begin 357 357 + (* It's a URL *) 358 358 + let n = match name_opt with 359 359 + | Some n -> n 360 360 + | None -> 361 361 + let base = Filename.basename pkg in 362 362 + if String.ends_with ~suffix:".git" base then 363 363 + String.sub base 0 (String.length base - 4) 364 364 + else base 365 365 + in 366 366 + (pkg, n) 367 367 + end else begin 368 368 + (* It's a package name - look up in repositories *) 369 369 + let repos = config.opam.repositories in 370 370 + if repos = [] then begin 371 371 + Format.eprintf "No repositories configured. Add one with: unpac opam repo add <name> <path>@."; 372 372 + exit 1 373 373 + end; 374 374 + match Unpac_opam.Repo.find_package ~repos ~name:pkg ?version:version_opt () with 375 375 + | None -> 376 376 + Format.eprintf "Package '%s' not found in configured repositories@." pkg; 377 377 + exit 1 378 378 + | Some result -> 379 379 + match result.metadata.dev_repo with 380 380 + | None -> 381 381 + Format.eprintf "Package '%s' has no dev-repo field@." pkg; 382 382 + exit 1 383 383 + | Some dev_repo -> 384 384 + (* Strip git+ prefix if present (opam dev-repo format) *) 385 385 + let url = if String.starts_with ~prefix:"git+" dev_repo then 386 386 + String.sub dev_repo 4 (String.length dev_repo - 4) 387 387 + else dev_repo in 388 388 + let n = match name_opt with Some n -> n | None -> pkg in 389 389 + (url, n) 390 390 + end 391 391 + in 392 392 + 393 393 + let info : Unpac.Backend.package_info = { 394 394 + name; 395 395 + url; 396 396 + branch = branch_opt; 397 397 + } in 398 398 + match Unpac_opam.Opam.add_package ~proc_mgr ~root ?cache info with 399 399 + | Unpac.Backend.Added { name = pkg_name; sha } -> 400 400 + Format.printf "Added %s (%s)@." pkg_name (String.sub sha 0 7); 401 401 + Format.printf "@.Next steps:@."; 402 402 + Format.printf " unpac opam edit %s # make local changes@." pkg_name; 403 403 + Format.printf " unpac opam merge %s <project> # merge into a project@." pkg_name 404 404 + | Unpac.Backend.Already_exists name -> 405 405 + Format.printf "Package %s already vendored@." name 406 406 + | Unpac.Backend.Failed { name; error } -> 407 407 + Format.eprintf "Error adding %s: %s@." name error; 408 408 + exit 1 409 409 + end 113 410 in 114 411 let info = Cmd.info "add" ~doc in 115 115 - Cmd.v info Term.(const run $ logging_term $ url_arg $ name_arg $ branch_arg) 412 412 + Cmd.v info Term.(const run $ logging_term $ pkg_arg $ name_arg $ version_arg $ branch_arg $ solve_arg $ cache_arg) 116 413 117 414 (* Opam list command *) 118 415 let opam_list_cmd = ··· 120 417 let run () = 121 418 with_root @@ fun ~env:_ ~fs:_ ~proc_mgr ~root -> 122 419 let packages = Unpac_opam.Opam.list_packages ~proc_mgr ~root in 123 123 - List.iter (Format.printf "%s@.") packages 420 420 + if packages = [] then begin 421 421 + Format.printf "No packages vendored@."; 422 422 + Format.printf "@.Hint: unpac opam add <package>@." 423 423 + end else 424 424 + List.iter (Format.printf "%s@.") packages 124 425 in 125 426 let info = Cmd.info "list" ~doc in 126 427 Cmd.v info Term.(const run $ logging_term) 127 428 429 429 + (* Opam edit command *) 430 430 + let opam_edit_cmd = 431 431 + let doc = "Open a package's patches worktree for editing." in 432 432 + let pkg_arg = 433 433 + let doc = "Package name to edit." in 434 434 + Arg.(required & pos 0 (some string) None & info [] ~docv:"PACKAGE" ~doc) 435 435 + in 436 436 + let run () pkg = 437 437 + with_root @@ fun ~env:_ ~fs:_ ~proc_mgr ~root -> 438 438 + (* Check package exists *) 439 439 + let packages = Unpac_opam.Opam.list_packages ~proc_mgr ~root in 440 440 + if not (List.mem pkg packages) then begin 441 441 + Format.eprintf "Package '%s' is not vendored@." pkg; 442 442 + exit 1 443 443 + end; 444 444 + (* Ensure patches worktree exists *) 445 445 + Unpac.Worktree.ensure ~proc_mgr root (Unpac.Worktree.Opam_patches pkg); 446 446 + let wt_path = Unpac.Worktree.path root (Unpac.Worktree.Opam_patches pkg) in 447 447 + let path_str = snd wt_path in 448 448 + Format.printf "Editing %s@." pkg; 449 449 + Format.printf "Worktree: %s@." path_str; 450 450 + Format.printf "@."; 451 451 + Format.printf "Make your changes, then:@."; 452 452 + Format.printf " cd %s@." path_str; 453 453 + Format.printf " git add -A && git commit -m 'your message'@."; 454 454 + Format.printf "@."; 455 455 + Format.printf "When done: unpac opam done %s@." pkg 456 456 + in 457 457 + let info = Cmd.info "edit" ~doc in 458 458 + Cmd.v info Term.(const run $ logging_term $ pkg_arg) 459 459 + 460 460 + (* Opam done command *) 461 461 + let opam_done_cmd = 462 462 + let doc = "Close a package's patches worktree after editing." in 463 463 + let pkg_arg = 464 464 + let doc = "Package name." in 465 465 + Arg.(required & pos 0 (some string) None & info [] ~docv:"PACKAGE" ~doc) 466 466 + in 467 467 + let run () pkg = 468 468 + with_root @@ fun ~env:_ ~fs:_ ~proc_mgr ~root -> 469 469 + let kind = Unpac.Worktree.Opam_patches pkg in 470 470 + if not (Unpac.Worktree.exists root kind) then begin 471 471 + Format.eprintf "No editing session for '%s'@." pkg; 472 472 + exit 1 473 473 + end; 474 474 + (* Check for uncommitted changes *) 475 475 + let wt_path = Unpac.Worktree.path root kind in 476 476 + let status = Unpac.Git.run_exn ~proc_mgr ~cwd:wt_path ["status"; "--porcelain"] in 477 477 + if String.trim status <> "" then begin 478 478 + Format.eprintf "Warning: uncommitted changes in %s@." pkg; 479 479 + Format.eprintf "Commit or discard them before closing.@."; 480 480 + exit 1 481 481 + end; 482 482 + (* Remove worktree *) 483 483 + Unpac.Worktree.remove ~proc_mgr root kind; 484 484 + Format.printf "Closed editing session for %s@." pkg; 485 485 + Format.printf "@.Next steps:@."; 486 486 + Format.printf " unpac opam diff %s # view your changes@." pkg; 487 487 + Format.printf " unpac opam merge %s <project> # merge into a project@." pkg 488 488 + in 489 489 + let info = Cmd.info "done" ~doc in 490 490 + Cmd.v info Term.(const run $ logging_term $ pkg_arg) 491 491 + 128 492 (* Opam update command *) 129 493 let opam_update_cmd = 130 494 let doc = "Update a vendored opam package from upstream." in ··· 135 499 let run () name = 136 500 with_root @@ fun ~env:_ ~fs:_ ~proc_mgr ~root -> 137 501 match Unpac_opam.Opam.update_package ~proc_mgr ~root name with 138 138 - | Unpac.Backend.Updated { name; old_sha; new_sha } -> 139 139 - Format.printf "Updated %s: %s -> %s@." name 140 140 - (String.sub old_sha 0 7) (String.sub new_sha 0 7) 502 502 + | Unpac.Backend.Updated { name = pkg_name; old_sha; new_sha } -> 503 503 + Format.printf "Updated %s: %s -> %s@." pkg_name 504 504 + (String.sub old_sha 0 7) (String.sub new_sha 0 7); 505 505 + Format.printf "@.Next steps:@."; 506 506 + Format.printf " unpac opam diff %s # view changes@." pkg_name; 507 507 + Format.printf " unpac opam merge %s <project> # merge into a project@." pkg_name 141 508 | Unpac.Backend.No_changes name -> 142 509 Format.printf "%s is up to date@." name 143 510 | Unpac.Backend.Update_failed { name; error } -> ··· 163 530 let patches_branch = Unpac_opam.Opam.patches_branch pkg in 164 531 match Unpac.Backend.merge_to_project ~proc_mgr ~root ~project ~patches_branch with 165 532 | Ok () -> 166 166 - Format.printf "Merged %s into project %s@." pkg project 533 533 + Format.printf "Merged %s into project %s@." pkg project; 534 534 + Format.printf "@.Next: Build your project in project/%s@." project 167 535 | Error (`Conflict files) -> 168 536 Format.eprintf "Merge conflict in %s:@." pkg; 169 537 List.iter (Format.eprintf " %s@.") files; ··· 173 541 let info = Cmd.info "merge" ~doc in 174 542 Cmd.v info Term.(const run $ logging_term $ pkg_arg $ project_arg) 175 543 544 544 + (* Opam info command *) 545 545 + let opam_info_cmd = 546 546 + let doc = "Show information about a vendored package." in 547 547 + let pkg_arg = 548 548 + let doc = "Package name." in 549 549 + Arg.(required & pos 0 (some string) None & info [] ~docv:"PACKAGE" ~doc) 550 550 + in 551 551 + let run () pkg = 552 552 + with_root @@ fun ~env:_ ~fs:_ ~proc_mgr ~root -> 553 553 + let git = Unpac.Worktree.git_dir root in 554 554 + (* Check package exists *) 555 555 + let packages = Unpac_opam.Opam.list_packages ~proc_mgr ~root in 556 556 + if not (List.mem pkg packages) then begin 557 557 + Format.eprintf "Package '%s' is not vendored@." pkg; 558 558 + exit 1 559 559 + end; 560 560 + (* Get remote URL *) 561 561 + let remote = "origin-" ^ pkg in 562 562 + let url = Unpac.Git.remote_url ~proc_mgr ~cwd:git remote in 563 563 + Format.printf "Package: %s@." pkg; 564 564 + (match url with 565 565 + | Some u -> Format.printf "URL: %s@." u 566 566 + | None -> ()); 567 567 + (* Get branch SHAs *) 568 568 + let upstream = Unpac_opam.Opam.upstream_branch pkg in 569 569 + let vendor = Unpac_opam.Opam.vendor_branch pkg in 570 570 + let patches = Unpac_opam.Opam.patches_branch pkg in 571 571 + (match Unpac.Git.rev_parse ~proc_mgr ~cwd:git upstream with 572 572 + | Some sha -> Format.printf "Upstream: %s@." (String.sub sha 0 7) 573 573 + | None -> ()); 574 574 + (match Unpac.Git.rev_parse ~proc_mgr ~cwd:git vendor with 575 575 + | Some sha -> Format.printf "Vendor: %s@." (String.sub sha 0 7) 576 576 + | None -> ()); 577 577 + (match Unpac.Git.rev_parse ~proc_mgr ~cwd:git patches with 578 578 + | Some sha -> Format.printf "Patches: %s@." (String.sub sha 0 7) 579 579 + | None -> ()); 580 580 + (* Count commits ahead *) 581 581 + let log_output = Unpac.Git.run_exn ~proc_mgr ~cwd:git 582 582 + ["log"; "--oneline"; vendor ^ ".." ^ patches] in 583 583 + let commits = List.length (String.split_on_char '\n' log_output |> 584 584 + List.filter (fun s -> String.trim s <> "")) in 585 585 + Format.printf "Local commits: %d@." commits; 586 586 + Format.printf "@.Commands:@."; 587 587 + Format.printf " unpac opam diff %s # view local changes@." pkg; 588 588 + Format.printf " unpac opam edit %s # edit package@." pkg; 589 589 + Format.printf " unpac opam update %s # fetch upstream@." pkg 590 590 + in 591 591 + let info = Cmd.info "info" ~doc in 592 592 + Cmd.v info Term.(const run $ logging_term $ pkg_arg) 593 593 + 594 594 + (* Opam diff command *) 595 595 + let opam_diff_cmd = 596 596 + let doc = "Show diff between vendor and patches branches." in 597 597 + let pkg_arg = 598 598 + let doc = "Package name." in 599 599 + Arg.(required & pos 0 (some string) None & info [] ~docv:"PACKAGE" ~doc) 600 600 + in 601 601 + let run () pkg = 602 602 + with_root @@ fun ~env:_ ~fs:_ ~proc_mgr ~root -> 603 603 + let git = Unpac.Worktree.git_dir root in 604 604 + (* Check package exists *) 605 605 + let packages = Unpac_opam.Opam.list_packages ~proc_mgr ~root in 606 606 + if not (List.mem pkg packages) then begin 607 607 + Format.eprintf "Package '%s' is not vendored@." pkg; 608 608 + exit 1 609 609 + end; 610 610 + let vendor = Unpac_opam.Opam.vendor_branch pkg in 611 611 + let patches = Unpac_opam.Opam.patches_branch pkg in 612 612 + let diff = Unpac.Git.run_exn ~proc_mgr ~cwd:git 613 613 + ["diff"; vendor; patches] in 614 614 + if String.trim diff = "" then begin 615 615 + Format.printf "No local changes@."; 616 616 + Format.printf "@.Hint: unpac opam edit %s # to make changes@." pkg 617 617 + end else begin 618 618 + print_string diff; 619 619 + Format.printf "@.Next: unpac opam merge %s <project>@." pkg 620 620 + end 621 621 + in 622 622 + let info = Cmd.info "diff" ~doc in 623 623 + Cmd.v info Term.(const run $ logging_term $ pkg_arg) 624 624 + 625 625 + (* Opam remove command *) 626 626 + let opam_remove_cmd = 627 627 + let doc = "Remove a vendored package." in 628 628 + let pkg_arg = 629 629 + let doc = "Package name to remove." in 630 630 + Arg.(required & pos 0 (some string) None & info [] ~docv:"PACKAGE" ~doc) 631 631 + in 632 632 + let run () pkg = 633 633 + with_root @@ fun ~env:_ ~fs:_ ~proc_mgr ~root -> 634 634 + let git = Unpac.Worktree.git_dir root in 635 635 + (* Check package exists *) 636 636 + let packages = Unpac_opam.Opam.list_packages ~proc_mgr ~root in 637 637 + if not (List.mem pkg packages) then begin 638 638 + Format.eprintf "Package '%s' is not vendored@." pkg; 639 639 + exit 1 640 640 + end; 641 641 + (* Remove worktrees if exist *) 642 642 + (try Unpac.Worktree.remove_force ~proc_mgr root (Unpac.Worktree.Opam_upstream pkg) with _ -> ()); 643 643 + (try Unpac.Worktree.remove_force ~proc_mgr root (Unpac.Worktree.Opam_vendor pkg) with _ -> ()); 644 644 + (try Unpac.Worktree.remove_force ~proc_mgr root (Unpac.Worktree.Opam_patches pkg) with _ -> ()); 645 645 + (* Delete branches *) 646 646 + let upstream = Unpac_opam.Opam.upstream_branch pkg in 647 647 + let vendor = Unpac_opam.Opam.vendor_branch pkg in 648 648 + let patches = Unpac_opam.Opam.patches_branch pkg in 649 649 + (try Unpac.Git.run_exn ~proc_mgr ~cwd:git ["branch"; "-D"; upstream] |> ignore with _ -> ()); 650 650 + (try Unpac.Git.run_exn ~proc_mgr ~cwd:git ["branch"; "-D"; vendor] |> ignore with _ -> ()); 651 651 + (try Unpac.Git.run_exn ~proc_mgr ~cwd:git ["branch"; "-D"; patches] |> ignore with _ -> ()); 652 652 + (* Remove remote *) 653 653 + let remote = "origin-" ^ pkg in 654 654 + (try Unpac.Git.run_exn ~proc_mgr ~cwd:git ["remote"; "remove"; remote] |> ignore with _ -> ()); 655 655 + Format.printf "Removed %s@." pkg; 656 656 + Format.printf "@.Hint: unpac opam add <package> # to add another package@." 657 657 + in 658 658 + let info = Cmd.info "remove" ~doc in 659 659 + Cmd.v info Term.(const run $ logging_term $ pkg_arg) 660 660 + 176 661 (* Opam command group *) 177 662 let opam_cmd = 178 663 let doc = "Opam package vendoring commands." in 179 664 let info = Cmd.info "opam" ~doc in 180 180 - Cmd.group info [opam_add_cmd; opam_list_cmd; opam_update_cmd; opam_merge_cmd] 665 665 + Cmd.group info [ 666 666 + opam_repo_cmd; 667 667 + opam_config_cmd; 668 668 + opam_add_cmd; 669 669 + opam_list_cmd; 670 670 + opam_edit_cmd; 671 671 + opam_done_cmd; 672 672 + opam_update_cmd; 673 673 + opam_merge_cmd; 674 674 + opam_info_cmd; 675 675 + opam_diff_cmd; 676 676 + opam_remove_cmd; 677 677 + ] 678 678 + 679 679 + (* Push command - push all unpac branches to a remote *) 680 680 + let push_cmd = 681 681 + let doc = "Push all unpac branches to a remote." in 682 682 + let remote_arg = 683 683 + let doc = "Remote name (e.g., origin)." in 684 684 + Arg.(required & pos 0 (some string) None & info [] ~docv:"REMOTE" ~doc) 685 685 + in 686 686 + let force_arg = 687 687 + let doc = "Force push (use with caution)." in 688 688 + Arg.(value & flag & info ["f"; "force"] ~doc) 689 689 + in 690 690 + let dry_run_arg = 691 691 + let doc = "Show what would be pushed without actually pushing." in 692 692 + Arg.(value & flag & info ["n"; "dry-run"] ~doc) 693 693 + in 694 694 + let run () remote force dry_run = 695 695 + with_root @@ fun ~env:_ ~fs:_ ~proc_mgr ~root -> 696 696 + let git = Unpac.Worktree.git_dir root in 697 697 + 698 698 + (* Check if remote exists *) 699 699 + (match Unpac.Git.remote_url ~proc_mgr ~cwd:git remote with 700 700 + | None -> 701 701 + Format.eprintf "Remote '%s' not configured.@." remote; 702 702 + Format.eprintf "Add it with: git -C %s remote add %s <url>@." (snd git) remote; 703 703 + exit 1 704 704 + | Some _ -> ()); 705 705 + 706 706 + (* Get all branches *) 707 707 + let all_branches = Unpac.Git.run_lines ~proc_mgr ~cwd:git ["branch"; "--format=%(refname:short)"] in 708 708 + 709 709 + (* Filter to only unpac-managed branches *) 710 710 + let unpac_branches = List.filter (fun b -> 711 711 + b = "main" || 712 712 + String.starts_with ~prefix:"opam/" b || 713 713 + String.starts_with ~prefix:"project/" b 714 714 + ) all_branches in 715 715 + 716 716 + if unpac_branches = [] then begin 717 717 + Format.printf "No branches to push@."; 718 718 + exit 0 719 719 + end; 720 720 + 721 721 + Format.printf "Branches to push to %s:@." remote; 722 722 + List.iter (fun b -> Format.printf " %s@." b) unpac_branches; 723 723 + Format.printf "@."; 724 724 + 725 725 + if dry_run then begin 726 726 + Format.printf "(dry run - no changes made)@." 727 727 + end else begin 728 728 + (* Build push command *) 729 729 + let force_flag = if force then ["--force"] else [] in 730 730 + let push_args = ["push"] @ force_flag @ [remote; "--"] @ unpac_branches in 731 731 + 732 732 + Format.printf "Pushing %d branches...@." (List.length unpac_branches); 733 733 + try 734 734 + Unpac.Git.run_exn ~proc_mgr ~cwd:git push_args |> ignore; 735 735 + Format.printf "Done.@." 736 736 + with e -> 737 737 + Format.eprintf "Push failed: %s@." (Printexc.to_string e); 738 738 + exit 1 739 739 + end 740 740 + in 741 741 + let info = Cmd.info "push" ~doc in 742 742 + Cmd.v info Term.(const run $ logging_term $ remote_arg $ force_arg $ dry_run_arg) 181 743 182 744 (* Main command *) 183 745 let main_cmd = 184 746 let doc = "Multi-backend vendoring tool using git worktrees." in 185 747 let info = Cmd.info "unpac" ~version:"0.1.0" ~doc in 186 186 - Cmd.group info [init_cmd; project_cmd; opam_cmd] 748 748 + Cmd.group info [init_cmd; project_cmd; opam_cmd; push_cmd] 187 749 188 750 let () = exit (Cmd.eval main_cmd)

+2 -4

dune-project

··· 26 26 unpac 27 27 opam-format 28 28 opam-core 29 29 - opam-repository 30 30 - opam-solver 31 31 - opam-0install-cudf 32 32 - cudf 29 29 + opam-state 30 30 + opam-0install 33 31 (cmdliner (>= 1.2.0))))

+55 -1

lib/config.ml

··· 24 24 25 25 type t = { 26 26 opam : opam_config; 27 27 + vendor_cache : string option; 27 28 projects : project_config list; 28 29 } 29 30 ··· 70 71 let codec : config Tomlt.t = 71 72 let open Tomlt in 72 73 let open Table in 73 73 - obj (fun opam : config -> { opam; projects = [] }) 74 74 + obj (fun opam vendor_cache : config -> { opam; vendor_cache; projects = [] }) 74 75 |> mem "opam" opam_config_codec ~enc:(fun (c : config) -> c.opam) 76 76 + |> opt_mem "vendor_cache" string ~enc:(fun (c : config) -> c.vendor_cache) 75 77 |> finish 76 78 77 79 (** {1 Loading} *) ··· 88 90 let load_exn path = 89 91 match load path with Ok c -> c | Error msg -> failwith msg 90 92 93 93 + (** {1 Saving} *) 94 94 + 95 95 + let save path config = 96 96 + try 97 97 + let content = Tomlt_bytesrw.encode_string codec config in 98 98 + Out_channel.with_open_text path (fun oc -> 99 99 + Out_channel.output_string oc content); 100 100 + Ok () 101 101 + with 102 102 + | Sys_error msg -> Error msg 103 103 + | Failure msg -> Error msg 104 104 + 105 105 + let save_exn path config = 106 106 + match save path config with 107 107 + | Ok () -> () 108 108 + | Error msg -> failwith msg 109 109 + 91 110 (** {1 Helpers} *) 92 111 112 112 + let empty_opam = { repositories = []; compiler = None } 113 113 + let empty = { opam = empty_opam; vendor_cache = None; projects = [] } 114 114 + 93 115 let find_project config name = 94 116 List.find_opt (fun p -> p.project_name = name) config.projects 117 117 + 118 118 + let add_repo config repo = 119 119 + let repos = config.opam.repositories @ [repo] in 120 120 + { config with opam = { config.opam with repositories = repos } } 121 121 + 122 122 + let remove_repo config name = 123 123 + let repos = List.filter (fun r -> r.repo_name <> name) config.opam.repositories in 124 124 + { config with opam = { config.opam with repositories = repos } } 125 125 + 126 126 + let find_repo config name = 127 127 + List.find_opt (fun r -> r.repo_name = name) config.opam.repositories 128 128 + 129 129 + let set_compiler config version = 130 130 + { config with opam = { config.opam with compiler = Some version } } 131 131 + 132 132 + let get_compiler config = 133 133 + config.opam.compiler 134 134 + 135 135 + let set_vendor_cache config path = 136 136 + { config with vendor_cache = Some path } 137 137 + 138 138 + let get_vendor_cache config = 139 139 + config.vendor_cache 140 140 + 141 141 + let resolve_vendor_cache ?cli_override config = 142 142 + (* Priority: CLI flag > env var > config file > default *) 143 143 + match cli_override with 144 144 + | Some path -> Some path 145 145 + | None -> 146 146 + match Sys.getenv_opt "UNPAC_VENDOR_CACHE" with 147 147 + | Some path -> Some path 148 148 + | None -> config.vendor_cache

+38

lib/config.mli

··· 24 24 25 25 type t = { 26 26 opam : opam_config; 27 27 + vendor_cache : string option; 27 28 projects : project_config list; 28 29 } 29 30 ··· 35 36 val load_exn : string -> t 36 37 (** [load_exn path] is like {!load} but raises on error. *) 37 38 39 39 + (** {1 Saving} *) 40 40 + 41 41 + val save : string -> t -> (unit, string) result 42 42 + (** [save path config] saves configuration to the TOML file at [path]. *) 43 43 + 44 44 + val save_exn : string -> t -> unit 45 45 + (** [save_exn path config] is like {!save} but raises on error. *) 46 46 + 38 47 (** {1 Helpers} *) 39 48 49 49 + val empty : t 50 50 + (** Empty configuration. *) 51 51 + 40 52 val find_project : t -> string -> project_config option 41 53 (** [find_project config name] finds a project by name. *) 54 54 + 55 55 + val add_repo : t -> repo_config -> t 56 56 + (** [add_repo config repo] adds a repository to the config. *) 57 57 + 58 58 + val remove_repo : t -> string -> t 59 59 + (** [remove_repo config name] removes a repository by name. *) 60 60 + 61 61 + val find_repo : t -> string -> repo_config option 62 62 + (** [find_repo config name] finds a repository by name. *) 63 63 + 64 64 + val set_compiler : t -> string -> t 65 65 + (** [set_compiler config version] sets the OCaml compiler version. *) 66 66 + 67 67 + val get_compiler : t -> string option 68 68 + (** [get_compiler config] gets the configured OCaml compiler version. *) 69 69 + 70 70 + val set_vendor_cache : t -> string -> t 71 71 + (** [set_vendor_cache config path] sets the vendor cache path. *) 72 72 + 73 73 + val get_vendor_cache : t -> string option 74 74 + (** [get_vendor_cache config] gets the configured vendor cache path. *) 75 75 + 76 76 + val resolve_vendor_cache : ?cli_override:string -> t -> string option 77 77 + (** [resolve_vendor_cache ?cli_override config] resolves vendor cache path. 78 78 + Priority: CLI flag > UNPAC_VENDOR_CACHE env var > config file. 79 79 + Returns None if not configured anywhere. *) 42 80 43 81 (** {1 Codecs} *) 44 82

+3 -3

lib/git.ml

··· 189 189 let rev_parse_short ~proc_mgr ~cwd ref_ = 190 190 run_exn ~proc_mgr ~cwd ["rev-parse"; "--short"; ref_] |> string_trim 191 191 192 192 - let ls_remote_default_branch ~proc_mgr ~url = 192 192 + let ls_remote_default_branch ~proc_mgr ~cwd ~url = 193 193 Log.info (fun m -> m "Detecting default branch for %s..." url); 194 194 (* Try to get the default branch from the remote *) 195 195 - let output = run_exn ~proc_mgr ["ls-remote"; "--symref"; url; "HEAD"] in 195 195 + let output = run_exn ~proc_mgr ~cwd ["ls-remote"; "--symref"; url; "HEAD"] in 196 196 (* Parse output like: ref: refs/heads/main\tHEAD *) 197 197 let default = 198 198 let lines = String.split_on_char '\n' output in ··· 217 217 (* Fallback: try common branch names *) 218 218 Log.debug (fun m -> m "Could not detect default branch, trying common names..."); 219 219 let try_branch name = 220 220 - match run ~proc_mgr ["ls-remote"; "--heads"; url; name] with 220 220 + match run ~proc_mgr ~cwd ["ls-remote"; "--heads"; url; name] with 221 221 | Ok output when String.trim output <> "" -> true 222 222 | _ -> false 223 223 in

+2 -1

lib/git.mli

··· 126 126 127 127 val ls_remote_default_branch : 128 128 proc_mgr:proc_mgr -> 129 129 + cwd:path -> 129 130 url:string -> 130 131 string 131 131 - (** [ls_remote_default_branch ~proc_mgr ~url] detects the default branch of remote. *) 132 132 + (** [ls_remote_default_branch ~proc_mgr ~cwd ~url] detects the default branch of remote. *) 132 133 133 134 val list_remotes : 134 135 proc_mgr:proc_mgr ->

+21 -1

lib/git_repo_lookup.ml

··· 1 1 (** Git repository URL lookup and rewriting. 2 2 3 3 This module handles URL rewriting for git repositories, mapping known 4 4 - slow upstream URLs to faster mirrors. *) 4 4 + slow upstream URLs to faster mirrors, and branch/tag overrides for 5 5 + specific packages. *) 5 6 6 7 (** Rewrite a git URL to use a faster mirror if available. 7 8 ··· 30 31 in 31 32 github_mirror ^ rest 32 33 else url 34 34 + 35 35 + (** Override branch/tag for specific packages. 36 36 + 37 37 + Some packages have unstable main branches or we want to pin to specific 38 38 + versions. This returns Some ref if an override exists, None otherwise. 39 39 + 40 40 + Currently handles: 41 41 + - dune: use tag 3.20.2 instead of main branch *) 42 42 + let branch_override ~name ~url = 43 43 + (* Dune's main branch can be unstable; pin to release tag *) 44 44 + let is_dune_url = 45 45 + String.equal url "https://github.com/ocaml/dune.git" || 46 46 + String.equal url "https://github.com/ocaml/dune" || 47 47 + String.equal url "git://github.com/ocaml/dune.git" 48 48 + in 49 49 + if name = "dune" || is_dune_url then 50 50 + Some "3.20.2" 51 51 + else 52 52 + None

+7 -1

lib/init.ml

··· 22 22 23 23 (** Initialize a new unpac project at the given path. *) 24 24 let init ~proc_mgr ~fs path = 25 25 - let root = Eio.Path.(fs / path) in 25 25 + (* Convert relative paths to absolute *) 26 26 + let abs_path = 27 27 + if Filename.is_relative path then 28 28 + Filename.concat (Sys.getcwd ()) path 29 29 + else path 30 30 + in 31 31 + let root = Eio.Path.(fs / abs_path) in 26 32 27 33 (* Create root directory *) 28 34 Eio.Path.mkdirs ~exists_ok:false ~perm:0o755 root;

-67

lib/opam/dev_repo.ml

··· 1 1 - type t = string 2 2 - 3 3 - let normalize_url s = 4 4 - let s = String.lowercase_ascii s in 5 5 - (* Remove git+ prefix *) 6 6 - let s = 7 7 - if String.starts_with ~prefix:"git+" s then 8 8 - String.sub s 4 (String.length s - 4) 9 9 - else s 10 10 - in 11 11 - (* Remove .git suffix *) 12 12 - let s = 13 13 - if String.ends_with ~suffix:".git" s then 14 14 - String.sub s 0 (String.length s - 4) 15 15 - else s 16 16 - in 17 17 - (* Remove trailing slash *) 18 18 - let s = 19 19 - if String.ends_with ~suffix:"/" s then 20 20 - String.sub s 0 (String.length s - 1) 21 21 - else s 22 22 - in 23 23 - (* Strip #branch fragment *) 24 24 - let s = 25 25 - match String.index_opt s '#' with 26 26 - | Some i -> String.sub s 0 i 27 27 - | None -> s 28 28 - in 29 29 - (* Normalize ssh-style github.com:user/repo to github.com/user/repo *) 30 30 - let s = 31 31 - match String.index_opt s ':' with 32 32 - | Some i when i > 0 -> 33 33 - let before = String.sub s 0 i in 34 34 - let after = String.sub s (i + 1) (String.length s - i - 1) in 35 35 - (* Only convert if it looks like host:path (no // after) *) 36 36 - if 37 37 - (not (String.contains before '/')) 38 38 - && not (String.starts_with ~prefix:"/" after) 39 39 - && String.contains before '.' 40 40 - then before ^ "/" ^ after 41 41 - else s 42 42 - | _ -> s 43 43 - in 44 44 - (* Remove protocol prefix for comparison *) 45 45 - let s = 46 46 - let protocols = [ "https://"; "http://"; "ssh://"; "git://"; "file://" ] in 47 47 - List.fold_left 48 48 - (fun s proto -> 49 49 - if String.starts_with ~prefix:proto s then 50 50 - String.sub s (String.length proto) (String.length s - String.length proto) 51 51 - else s) 52 52 - s protocols 53 53 - in 54 54 - s 55 55 - 56 56 - let of_opam_url url = normalize_url (OpamUrl.to_string url) 57 57 - 58 58 - let of_string s = normalize_url s 59 59 - 60 60 - let equal = String.equal 61 61 - let compare = String.compare 62 62 - let to_string t = t 63 63 - 64 64 - let pp fmt t = Format.pp_print_string fmt t 65 65 - 66 66 - module Map = Map.Make (String) 67 67 - module Set = Set.Make (String)

-47

lib/opam/dev_repo.mli

··· 1 1 - (** Normalized dev-repo URLs. 2 2 - 3 3 - This module provides URL normalization for dev-repo fields to enable 4 4 - matching packages that share the same source repository even when 5 5 - the URLs are written differently. 6 6 - 7 7 - Normalization rules: 8 8 - - Strip [.git] suffix 9 9 - - Normalize to lowercase 10 10 - - Remove [git+] prefix from transport 11 11 - - Normalize [github.com:user/repo] to [github.com/user/repo] 12 12 - - Remove trailing slashes 13 13 - - Strip [#branch] fragment *) 14 14 - 15 15 - (** {1 Types} *) 16 16 - 17 17 - type t 18 18 - (** Normalized dev-repo URL. *) 19 19 - 20 20 - (** {1 Creation} *) 21 21 - 22 22 - val of_opam_url : OpamUrl.t -> t 23 23 - (** [of_opam_url url] creates a normalized dev-repo from an opam URL. *) 24 24 - 25 25 - val of_string : string -> t 26 26 - (** [of_string s] parses and normalizes a URL string. *) 27 27 - 28 28 - (** {1 Comparison} *) 29 29 - 30 30 - val equal : t -> t -> bool 31 31 - (** [equal a b] is [true] if [a] and [b] represent the same repository. *) 32 32 - 33 33 - val compare : t -> t -> int 34 34 - (** [compare a b] is a total ordering on normalized URLs. *) 35 35 - 36 36 - (** {1 Conversion} *) 37 37 - 38 38 - val to_string : t -> string 39 39 - (** [to_string t] returns the normalized URL string. *) 40 40 - 41 41 - val pp : Format.formatter -> t -> unit 42 42 - (** [pp fmt t] pretty-prints the normalized URL. *) 43 43 - 44 44 - (** {1 Collections} *) 45 45 - 46 46 - module Map : Map.S with type key = t 47 47 - module Set : Set.S with type elt = t

+2 -6

lib/opam/dune

··· 4 4 (libraries 5 5 unpac 6 6 cmdliner 7 7 - jsont 8 8 - jsont.bytesrw 9 7 opam-format 10 8 opam-core 11 11 - opam-repository 12 12 - opam-solver 13 13 - opam-0install-cudf 14 14 - cudf)) 9 9 + opam-state 10 10 + opam-0install))

+33 -15

lib/opam/opam.ml

··· 8 8 module Worktree = Unpac.Worktree 9 9 module Git = Unpac.Git 10 10 module Git_repo_lookup = Unpac.Git_repo_lookup 11 11 + module Vendor_cache = Unpac.Vendor_cache 11 12 module Backend = Unpac.Backend 12 13 13 14 let name = "opam" ··· 61 62 end 62 63 ) 63 64 64 64 - let add_package ~proc_mgr ~root (info : Backend.package_info) = 65 65 + let add_package ~proc_mgr ~root ?cache (info : Backend.package_info) = 65 66 let pkg = info.name in 66 67 let git = Worktree.git_dir root in 67 68 ··· 73 74 (* Step 1: Create upstream branch and fetch *) 74 75 let upstream_wt = Worktree.path root (upstream_kind pkg) in 75 76 76 76 - (* Add remote for this package *) 77 77 - let remote = "origin-" ^ pkg in 77 77 + (* Rewrite URL for known mirrors *) 78 78 let url = Git_repo_lookup.rewrite_url info.url in 79 79 - ignore (Git.ensure_remote ~proc_mgr ~cwd:git ~name:remote ~url); 80 79 81 81 - (* Fetch from remote *) 82 82 - Git.fetch ~proc_mgr ~cwd:git ~remote; 83 83 - 84 84 - (* Determine the ref to use *) 80 80 + (* Determine the ref to use: explicit > override > default *) 85 81 let branch = match info.branch with 86 82 | Some b -> b 87 87 - | None -> Git.ls_remote_default_branch ~proc_mgr ~url 83 83 + | None -> 84 84 + match Git_repo_lookup.branch_override ~name:pkg ~url with 85 85 + | Some b -> b 86 86 + | None -> Git.ls_remote_default_branch ~proc_mgr ~cwd:git ~url 88 87 in 89 89 - let ref_point = remote ^ "/" ^ branch in 88 88 + 89 89 + (* Fetch - either via cache or directly *) 90 90 + let ref_point = match cache with 91 91 + | Some cache_path -> 92 92 + (* Fetch through vendor cache *) 93 93 + Vendor_cache.fetch_to_project ~proc_mgr 94 94 + ~cache:cache_path ~project_git:git ~url ~branch 95 95 + | None -> 96 96 + (* Direct fetch *) 97 97 + let remote = "origin-" ^ pkg in 98 98 + ignore (Git.ensure_remote ~proc_mgr ~cwd:git ~name:remote ~url); 99 99 + Git.fetch ~proc_mgr ~cwd:git ~remote; 100 100 + remote ^ "/" ^ branch 101 101 + in 90 102 91 103 (* Create upstream branch *) 92 104 Git.branch_force ~proc_mgr ~cwd:git ··· 129 141 (try Worktree.remove_force ~proc_mgr root (vendor_kind pkg) with _ -> ()); 130 142 Backend.Failed { name = pkg; error = Printexc.to_string exn } 131 143 132 132 - let update_package ~proc_mgr ~root pkg = 144 144 + let update_package ~proc_mgr ~root ?cache pkg = 133 145 let git = Worktree.git_dir root in 134 146 135 147 try ··· 137 149 if not (Worktree.branch_exists ~proc_mgr root (patches_kind pkg)) then 138 150 Backend.Update_failed { name = pkg; error = "Package not vendored" } 139 151 else begin 140 140 - (* Get remote URL *) 152 152 + (* Get remote URL - check vendor-cache remote first, then origin-<pkg> *) 141 153 let remote = "origin-" ^ pkg in 142 154 let url = match Git.remote_url ~proc_mgr ~cwd:git remote with 143 155 | Some u -> u 144 156 | None -> failwith ("Remote not found: " ^ remote) 145 157 in 146 158 147 147 - (* Fetch latest *) 148 148 - Git.fetch ~proc_mgr ~cwd:git ~remote; 159 159 + (* Fetch latest - either via cache or directly *) 160 160 + (match cache with 161 161 + | Some cache_path -> 162 162 + let branch = Git.ls_remote_default_branch ~proc_mgr ~cwd:git ~url in 163 163 + ignore (Vendor_cache.fetch_to_project ~proc_mgr 164 164 + ~cache:cache_path ~project_git:git ~url ~branch) 165 165 + | None -> 166 166 + Git.fetch ~proc_mgr ~cwd:git ~remote); 149 167 150 168 (* Get old SHA *) 151 169 let old_sha = match Git.rev_parse ~proc_mgr ~cwd:git (upstream_branch pkg) with ··· 154 172 in 155 173 156 174 (* Determine default branch and update upstream *) 157 157 - let default_branch = Git.ls_remote_default_branch ~proc_mgr ~url in 175 175 + let default_branch = Git.ls_remote_default_branch ~proc_mgr ~cwd:git ~url in 158 176 let ref_point = remote ^ "/" ^ default_branch in 159 177 Git.branch_force ~proc_mgr ~cwd:git 160 178 ~name:(upstream_branch pkg) ~point:ref_point;

+12 -6

lib/opam/opam.mli

··· 33 33 val add_package : 34 34 proc_mgr:Unpac.Git.proc_mgr -> 35 35 root:Unpac.Worktree.root -> 36 36 + ?cache:Unpac.Vendor_cache.t -> 36 37 Unpac.Backend.package_info -> 37 38 Unpac.Backend.add_result 38 38 - (** [add_package ~proc_mgr ~root info] vendors a single package. 39 39 + (** [add_package ~proc_mgr ~root ?cache info] vendors a single package. 39 40 40 40 - 1. Fetches upstream into opam/upstream/<pkg> 41 41 + 1. Fetches upstream into opam/upstream/<pkg> (via cache if provided) 41 42 2. Creates opam/vendor/<pkg> orphan with vendor/opam/<pkg>/ prefix 42 42 - 3. Creates opam/patches/<pkg> from vendor *) 43 43 + 3. Creates opam/patches/<pkg> from vendor 44 44 + 45 45 + @param cache Optional vendor cache for shared fetches across projects. *) 43 46 44 47 val update_package : 45 48 proc_mgr:Unpac.Git.proc_mgr -> 46 49 root:Unpac.Worktree.root -> 50 50 + ?cache:Unpac.Vendor_cache.t -> 47 51 string -> 48 52 Unpac.Backend.update_result 49 49 - (** [update_package ~proc_mgr ~root name] updates a package from upstream. 53 53 + (** [update_package ~proc_mgr ~root ?cache name] updates a package from upstream. 50 54 51 51 - 1. Fetches latest into opam/upstream/<pkg> 55 55 + 1. Fetches latest into opam/upstream/<pkg> (via cache if provided) 52 56 2. Updates opam/vendor/<pkg> with new content 53 57 54 54 - Does NOT rebase patches - call [Backend.rebase_patches] separately. *) 58 58 + Does NOT rebase patches - call [Backend.rebase_patches] separately. 59 59 + 60 60 + @param cache Optional vendor cache for shared fetches across projects. *) 55 61 56 62 val list_packages : 57 63 proc_mgr:Unpac.Git.proc_mgr ->

+107

lib/opam/opam_file.ml

··· 1 1 + (** Opam file parsing for extracting package metadata. *) 2 2 + 3 3 + type metadata = { 4 4 + name : string; 5 5 + version : string; 6 6 + dev_repo : string option; 7 7 + synopsis : string option; 8 8 + } 9 9 + 10 10 + let empty_metadata = { 11 11 + name = ""; 12 12 + version = ""; 13 13 + dev_repo = None; 14 14 + synopsis = None; 15 15 + } 16 16 + 17 17 + (** Parse an opam file and extract metadata. *) 18 18 + let parse ~name ~version content = 19 19 + try 20 20 + let opam = OpamParser.FullPos.string content "<opam>" in 21 21 + let items = opam.file_contents in 22 22 + 23 23 + let dev_repo = ref None in 24 24 + let synopsis = ref None in 25 25 + 26 26 + List.iter (fun item -> 27 27 + match item.OpamParserTypes.FullPos.pelem with 28 28 + | OpamParserTypes.FullPos.Variable (name_pos, value_pos) -> 29 29 + let var_name = name_pos.OpamParserTypes.FullPos.pelem in 30 30 + (match var_name, value_pos.OpamParserTypes.FullPos.pelem with 31 31 + | "dev-repo", OpamParserTypes.FullPos.String s -> 32 32 + dev_repo := Some s 33 33 + | "synopsis", OpamParserTypes.FullPos.String s -> 34 34 + synopsis := Some s 35 35 + | _ -> ()) 36 36 + | _ -> () 37 37 + ) items; 38 38 + 39 39 + { name; version; dev_repo = !dev_repo; synopsis = !synopsis } 40 40 + with _ -> 41 41 + { empty_metadata with name; version } 42 42 + 43 43 + (** Parse an opam file from a path. *) 44 44 + let parse_file ~name ~version path = 45 45 + let content = In_channel.with_open_text path In_channel.input_all in 46 46 + parse ~name ~version content 47 47 + 48 48 + (** Find a package in an opam repository directory. 49 49 + Returns the path to the opam file if found. *) 50 50 + let find_in_repo ~repo_path ~name ?version () = 51 51 + let packages_dir = Filename.concat repo_path "packages" in 52 52 + let pkg_dir = Filename.concat packages_dir name in 53 53 + 54 54 + if not (Sys.file_exists pkg_dir && Sys.is_directory pkg_dir) then 55 55 + None 56 56 + else 57 57 + (* List version directories *) 58 58 + let entries = Sys.readdir pkg_dir |> Array.to_list in 59 59 + let version_dirs = List.filter (fun entry -> 60 60 + let full = Filename.concat pkg_dir entry in 61 61 + Sys.is_directory full && String.starts_with ~prefix:(name ^ ".") entry 62 62 + ) entries in 63 63 + 64 64 + match version with 65 65 + | Some v -> 66 66 + (* Look for specific version *) 67 67 + let target = name ^ "." ^ v in 68 68 + if List.mem target version_dirs then 69 69 + let opam_path = Filename.concat (Filename.concat pkg_dir target) "opam" in 70 70 + if Sys.file_exists opam_path then Some (opam_path, v) 71 71 + else None 72 72 + else None 73 73 + | None -> 74 74 + (* Find latest version (simple string sort, works for semver) *) 75 75 + let sorted = List.sort (fun a b -> String.compare b a) version_dirs in 76 76 + match sorted with 77 77 + | [] -> None 78 78 + | latest :: _ -> 79 79 + let v = String.sub latest (String.length name + 1) 80 80 + (String.length latest - String.length name - 1) in 81 81 + let opam_path = Filename.concat (Filename.concat pkg_dir latest) "opam" in 82 82 + if Sys.file_exists opam_path then Some (opam_path, v) 83 83 + else None 84 84 + 85 85 + (** Get metadata for a package from an opam repository. *) 86 86 + let get_metadata ~repo_path ~name ?version () = 87 87 + match find_in_repo ~repo_path ~name ?version () with 88 88 + | None -> None 89 89 + | Some (opam_path, v) -> 90 90 + Some (parse_file ~name ~version:v opam_path) 91 91 + 92 92 + (** List all versions of a package in a repository. *) 93 93 + let list_versions ~repo_path ~name = 94 94 + let packages_dir = Filename.concat repo_path "packages" in 95 95 + let pkg_dir = Filename.concat packages_dir name in 96 96 + 97 97 + if not (Sys.file_exists pkg_dir && Sys.is_directory pkg_dir) then 98 98 + [] 99 99 + else 100 100 + Sys.readdir pkg_dir 101 101 + |> Array.to_list 102 102 + |> List.filter_map (fun entry -> 103 103 + if String.starts_with ~prefix:(name ^ ".") entry then 104 104 + Some (String.sub entry (String.length name + 1) 105 105 + (String.length entry - String.length name - 1)) 106 106 + else None) 107 107 + |> List.sort String.compare

+24

lib/opam/opam_file.mli

··· 1 1 + (** Opam file parsing for extracting package metadata. *) 2 2 + 3 3 + type metadata = { 4 4 + name : string; 5 5 + version : string; 6 6 + dev_repo : string option; 7 7 + synopsis : string option; 8 8 + } 9 9 + 10 10 + val parse : name:string -> version:string -> string -> metadata 11 11 + (** [parse ~name ~version content] parses opam file content. *) 12 12 + 13 13 + val parse_file : name:string -> version:string -> string -> metadata 14 14 + (** [parse_file ~name ~version path] parses an opam file from disk. *) 15 15 + 16 16 + val find_in_repo : repo_path:string -> name:string -> ?version:string -> unit -> (string * string) option 17 17 + (** [find_in_repo ~repo_path ~name ?version ()] finds a package in an opam repository. 18 18 + Returns [Some (opam_file_path, version)] if found. *) 19 19 + 20 20 + val get_metadata : repo_path:string -> name:string -> ?version:string -> unit -> metadata option 21 21 + (** [get_metadata ~repo_path ~name ?version ()] gets package metadata from a repository. *) 22 22 + 23 23 + val list_versions : repo_path:string -> name:string -> string list 24 24 + (** [list_versions ~repo_path ~name] lists all available versions of a package. *)

+71

lib/opam/repo.ml

··· 1 1 + (** Opam repository operations. *) 2 2 + 3 3 + type repo = { 4 4 + name : string; 5 5 + path : string; 6 6 + } 7 7 + 8 8 + type search_result = { 9 9 + repo : repo; 10 10 + metadata : Opam_file.metadata; 11 11 + } 12 12 + 13 13 + (** Resolve repository path from config. *) 14 14 + let resolve_repo (cfg : Unpac.Config.repo_config) : repo option = 15 15 + match cfg.source with 16 16 + | Unpac.Config.Local path -> 17 17 + if Sys.file_exists path && Sys.is_directory path then 18 18 + Some { name = cfg.repo_name; path } 19 19 + else None 20 20 + | Unpac.Config.Remote _url -> 21 21 + (* Remote repos not yet supported *) 22 22 + None 23 23 + 24 24 + (** Search for a package in configured repositories. *) 25 25 + let find_package ~repos ~name ?version () : search_result option = 26 26 + let rec search = function 27 27 + | [] -> None 28 28 + | cfg :: rest -> 29 29 + match resolve_repo cfg with 30 30 + | None -> search rest 31 31 + | Some repo -> 32 32 + match Opam_file.get_metadata ~repo_path:repo.path ~name ?version () with 33 33 + | None -> search rest 34 34 + | Some metadata -> Some { repo; metadata } 35 35 + in 36 36 + search repos 37 37 + 38 38 + (** List all versions of a package across repositories. *) 39 39 + let list_versions ~repos ~name : (repo * string list) list = 40 40 + List.filter_map (fun cfg -> 41 41 + match resolve_repo cfg with 42 42 + | None -> None 43 43 + | Some repo -> 44 44 + let versions = Opam_file.list_versions ~repo_path:repo.path ~name in 45 45 + if versions = [] then None 46 46 + else Some (repo, versions) 47 47 + ) repos 48 48 + 49 49 + (** Search for packages matching a pattern. *) 50 50 + let search_packages ~repos ~pattern : (repo * string) list = 51 51 + List.concat_map (fun cfg -> 52 52 + match resolve_repo cfg with 53 53 + | None -> [] 54 54 + | Some repo -> 55 55 + let packages_dir = Filename.concat repo.path "packages" in 56 56 + if not (Sys.file_exists packages_dir) then [] 57 57 + else 58 58 + Sys.readdir packages_dir 59 59 + |> Array.to_list 60 60 + |> List.filter (fun name -> 61 61 + (* Simple substring match *) 62 62 + let pattern_lower = String.lowercase_ascii pattern in 63 63 + let name_lower = String.lowercase_ascii name in 64 64 + String.length pattern_lower <= String.length name_lower && 65 65 + (let rec check i = 66 66 + if i > String.length name_lower - String.length pattern_lower then false 67 67 + else if String.sub name_lower i (String.length pattern_lower) = pattern_lower then true 68 68 + else check (i + 1) 69 69 + in check 0)) 70 70 + |> List.map (fun name -> (repo, name)) 71 71 + ) repos

+32

lib/opam/repo.mli

··· 1 1 + (** Opam repository operations. *) 2 2 + 3 3 + type repo = { 4 4 + name : string; 5 5 + path : string; 6 6 + } 7 7 + 8 8 + type search_result = { 9 9 + repo : repo; 10 10 + metadata : Opam_file.metadata; 11 11 + } 12 12 + 13 13 + val find_package : 14 14 + repos:Unpac.Config.repo_config list -> 15 15 + name:string -> 16 16 + ?version:string -> 17 17 + unit -> 18 18 + search_result option 19 19 + (** [find_package ~repos ~name ?version ()] searches for a package in repositories. 20 20 + Returns the first match found. *) 21 21 + 22 22 + val list_versions : 23 23 + repos:Unpac.Config.repo_config list -> 24 24 + name:string -> 25 25 + (repo * string list) list 26 26 + (** [list_versions ~repos ~name] lists all versions across repositories. *) 27 27 + 28 28 + val search_packages : 29 29 + repos:Unpac.Config.repo_config list -> 30 30 + pattern:string -> 31 31 + (repo * string) list 32 32 + (** [search_packages ~repos ~pattern] searches for packages matching a pattern. *)

-120

lib/opam/repo_index.ml

··· 1 1 - type package_info = { 2 2 - name : OpamPackage.Name.t; 3 3 - version : OpamPackage.Version.t; 4 4 - opam : OpamFile.OPAM.t; 5 5 - dev_repo : Dev_repo.t option; 6 6 - source_repo : string; 7 7 - } 8 8 - 9 9 - type t = { 10 10 - packages : package_info OpamPackage.Map.t; 11 11 - by_name : OpamPackage.Set.t OpamPackage.Name.Map.t; 12 12 - by_dev_repo : OpamPackage.Set.t Dev_repo.Map.t; 13 13 - repos : string list; 14 14 - } 15 15 - 16 16 - let empty = 17 17 - { 18 18 - packages = OpamPackage.Map.empty; 19 19 - by_name = OpamPackage.Name.Map.empty; 20 20 - by_dev_repo = Dev_repo.Map.empty; 21 21 - repos = []; 22 22 - } 23 23 - 24 24 - let add_package nv info t = 25 25 - let packages = OpamPackage.Map.add nv info t.packages in 26 26 - let by_name = 27 27 - let name = OpamPackage.name nv in 28 28 - let existing = 29 29 - match OpamPackage.Name.Map.find_opt name t.by_name with 30 30 - | Some s -> s 31 31 - | None -> OpamPackage.Set.empty 32 32 - in 33 33 - OpamPackage.Name.Map.add name (OpamPackage.Set.add nv existing) t.by_name 34 34 - in 35 35 - let by_dev_repo = 36 36 - match info.dev_repo with 37 37 - | Some dev_repo -> 38 38 - let existing = 39 39 - match Dev_repo.Map.find_opt dev_repo t.by_dev_repo with 40 40 - | Some s -> s 41 41 - | None -> OpamPackage.Set.empty 42 42 - in 43 43 - Dev_repo.Map.add dev_repo (OpamPackage.Set.add nv existing) t.by_dev_repo 44 44 - | None -> t.by_dev_repo 45 45 - in 46 46 - { t with packages; by_name; by_dev_repo } 47 47 - 48 48 - let load_local_repo ~name ~path t = 49 49 - let repo_root = OpamFilename.Dir.of_string path in 50 50 - let pkg_prefixes = OpamRepository.packages_with_prefixes repo_root in 51 51 - let t = 52 52 - if List.mem name t.repos then t else { t with repos = name :: t.repos } 53 53 - in 54 54 - OpamPackage.Map.fold 55 55 - (fun nv prefix acc -> 56 56 - let opam_file = OpamRepositoryPath.opam repo_root prefix nv in 57 57 - match OpamFile.OPAM.read_opt opam_file with 58 58 - | Some opam -> 59 59 - let dev_repo = 60 60 - OpamFile.OPAM.dev_repo opam |> Option.map Dev_repo.of_opam_url 61 61 - in 62 62 - let info = 63 63 - { 64 64 - name = OpamPackage.name nv; 65 65 - version = OpamPackage.version nv; 66 66 - opam; 67 67 - dev_repo; 68 68 - source_repo = name; 69 69 - } 70 70 - in 71 71 - add_package nv info acc 72 72 - | None -> acc) 73 73 - pkg_prefixes t 74 74 - 75 75 - let all_packages t = 76 76 - OpamPackage.Map.fold (fun _ info acc -> info :: acc) t.packages [] 77 77 - 78 78 - let find_package name t = 79 79 - match OpamPackage.Name.Map.find_opt name t.by_name with 80 80 - | None -> [] 81 81 - | Some nvs -> 82 82 - OpamPackage.Set.fold 83 83 - (fun nv acc -> 84 84 - match OpamPackage.Map.find_opt nv t.packages with 85 85 - | Some info -> info :: acc 86 86 - | None -> acc) 87 87 - nvs [] 88 88 - 89 89 - let find_package_version name version t = 90 90 - let nv = OpamPackage.create name version in 91 91 - OpamPackage.Map.find_opt nv t.packages 92 92 - 93 93 - let packages_by_dev_repo dev_repo t = 94 94 - match Dev_repo.Map.find_opt dev_repo t.by_dev_repo with 95 95 - | None -> [] 96 96 - | Some nvs -> 97 97 - OpamPackage.Set.fold 98 98 - (fun nv acc -> 99 99 - match OpamPackage.Map.find_opt nv t.packages with 100 100 - | Some info -> info :: acc 101 101 - | None -> acc) 102 102 - nvs [] 103 103 - 104 104 - let related_packages name t = 105 105 - let versions = find_package name t in 106 106 - let dev_repos = 107 107 - List.filter_map (fun info -> info.dev_repo) versions 108 108 - |> List.sort_uniq Dev_repo.compare 109 109 - in 110 110 - List.concat_map (fun dr -> packages_by_dev_repo dr t) dev_repos 111 111 - |> List.sort_uniq (fun a b -> 112 112 - let cmp = OpamPackage.Name.compare a.name b.name in 113 113 - if cmp <> 0 then cmp 114 114 - else OpamPackage.Version.compare a.version b.version) 115 115 - 116 116 - let package_names t = 117 117 - OpamPackage.Name.Map.fold (fun name _ acc -> name :: acc) t.by_name [] 118 118 - 119 119 - let package_count t = OpamPackage.Map.cardinal t.packages 120 120 - let repo_count t = List.length t.repos

-59

lib/opam/repo_index.mli

··· 1 1 - (** Repository index for opam packages. 2 2 - 3 3 - This module provides functionality to load and query packages from 4 4 - multiple opam repositories, with support for merging with configurable 5 5 - priority. *) 6 6 - 7 7 - (** {1 Types} *) 8 8 - 9 9 - type package_info = { 10 10 - name : OpamPackage.Name.t; 11 11 - version : OpamPackage.Version.t; 12 12 - opam : OpamFile.OPAM.t; 13 13 - dev_repo : Dev_repo.t option; 14 14 - source_repo : string; (** Name of the repository this package came from *) 15 15 - } 16 16 - (** Information about a single package version. *) 17 17 - 18 18 - type t 19 19 - (** The repository index containing all loaded packages. *) 20 20 - 21 21 - (** {1 Creation} *) 22 22 - 23 23 - val empty : t 24 24 - (** [empty] is an empty repository index. *) 25 25 - 26 26 - val load_local_repo : name:string -> path:string -> t -> t 27 27 - (** [load_local_repo ~name ~path index] loads all packages from the local 28 28 - opam repository at [path] and adds them to [index]. Packages from this 29 29 - load will take priority over existing packages with the same name/version. *) 30 30 - 31 31 - (** {1 Queries} *) 32 32 - 33 33 - val all_packages : t -> package_info list 34 34 - (** [all_packages t] returns all packages in the index. *) 35 35 - 36 36 - val find_package : OpamPackage.Name.t -> t -> package_info list 37 37 - (** [find_package name t] returns all versions of package [name]. *) 38 38 - 39 39 - val find_package_version : 40 40 - OpamPackage.Name.t -> OpamPackage.Version.t -> t -> package_info option 41 41 - (** [find_package_version name version t] returns the specific package version. *) 42 42 - 43 43 - val packages_by_dev_repo : Dev_repo.t -> t -> package_info list 44 44 - (** [packages_by_dev_repo dev_repo t] returns all packages with the given dev-repo. *) 45 45 - 46 46 - val related_packages : OpamPackage.Name.t -> t -> package_info list 47 47 - (** [related_packages name t] returns all packages that share a dev-repo with 48 48 - any version of package [name]. *) 49 49 - 50 50 - val package_names : t -> OpamPackage.Name.t list 51 51 - (** [package_names t] returns all unique package names. *) 52 52 - 53 53 - (** {1 Statistics} *) 54 54 - 55 55 - val package_count : t -> int 56 56 - (** [package_count t] returns the total number of package versions. *) 57 57 - 58 58 - val repo_count : t -> int 59 59 - (** [repo_count t] returns the number of source repositories loaded. *)

+149 -411

lib/opam/solver.ml

··· 1 1 - open Cmdliner 2 2 - 3 3 - type version_constraint = OpamFormula.relop * OpamPackage.Version.t 4 4 - 5 5 - type package_spec = { 6 6 - name : OpamPackage.Name.t; 7 7 - constraint_ : version_constraint option; 8 8 - } 9 9 - 10 10 - (* Target platform configuration *) 11 11 - type platform = { 12 12 - os : string; 13 13 - os_family : string; 14 14 - os_distribution : string; 15 15 - arch : string; 16 16 - } 17 17 - 18 18 - let debian_x86_64 = { 19 19 - os = "linux"; 20 20 - os_family = "debian"; 21 21 - os_distribution = "debian"; 22 22 - arch = "x86_64"; 23 23 - } 24 24 - 25 25 - (* Create a filter environment for the target platform *) 26 26 - let make_filter_env platform : OpamFilter.env = 27 27 - fun var -> 28 28 - let open OpamVariable in 29 29 - let s = to_string (Full.variable var) in 30 30 - match s with 31 31 - | "os" -> Some (S platform.os) 32 32 - | "os-family" -> Some (S platform.os_family) 33 33 - | "os-distribution" -> Some (S platform.os_distribution) 34 34 - | "arch" -> Some (S platform.arch) 35 35 - | "opam-version" -> Some (S "2.1.0") 36 36 - | "make" -> Some (S "make") 37 37 - | "jobs" -> Some (S "4") 38 38 - | "pinned" -> Some (B false) 39 39 - | "build" -> Some (B true) 40 40 - | "post" -> Some (B false) 41 41 - | "dev" -> Some (B false) 42 42 - | "with-test" -> Some (B false) 43 43 - | "with-doc" -> Some (B false) 44 44 - | "with-dev-setup" -> Some (B false) 45 45 - | _ -> None 46 46 - 47 47 - (* Check if a package is available on the target platform *) 48 48 - let is_available_on_platform env (opam : OpamFile.OPAM.t) : bool = 49 49 - let available = OpamFile.OPAM.available opam in 50 50 - OpamFilter.opt_eval_to_bool env (Some available) 51 51 - 52 52 - (* Check if a package has the compiler flag or is a compiler-related package *) 53 53 - let is_compiler_package (opam : OpamFile.OPAM.t) (name : OpamPackage.Name.t) : bool = 54 54 - let name_s = OpamPackage.Name.to_string name in 55 55 - (* Check for flags:compiler *) 56 56 - let has_compiler_flag = 57 57 - List.mem OpamTypes.Pkgflag_Compiler (OpamFile.OPAM.flags opam) 58 58 - in 59 59 - (* Also filter out known compiler-related packages by name pattern *) 60 60 - let is_compiler_name = 61 61 - name_s = "ocaml" || 62 62 - String.starts_with ~prefix:"ocaml-base-compiler" name_s || 63 63 - String.starts_with ~prefix:"ocaml-variants" name_s || 64 64 - String.starts_with ~prefix:"ocaml-system" name_s || 65 65 - String.starts_with ~prefix:"ocaml-option-" name_s || 66 66 - String.starts_with ~prefix:"ocaml-config" name_s || 67 67 - String.starts_with ~prefix:"ocaml-compiler" name_s || 68 68 - String.starts_with ~prefix:"base-" name_s || (* base-threads, base-unix, etc. *) 69 69 - String.starts_with ~prefix:"dkml-base-compiler" name_s || 70 70 - String.starts_with ~prefix:"dkml-runtime" name_s 71 71 - in 72 72 - has_compiler_flag || is_compiler_name 73 73 - 74 74 - (* Filter dependencies to remove platform-filtered ones. 75 75 - Uses OpamFilter.filter_formula which evaluates filters and simplifies. *) 76 76 - let filter_depends env (formula : OpamTypes.filtered_formula) : OpamTypes.formula = 77 77 - OpamFilter.filter_formula ~default:false env formula 78 78 - 79 79 - (* Parse version constraint from string like ">=1.0.0" *) 80 80 - let parse_constraint s = 81 81 - let s = String.trim s in 82 82 - if String.length s = 0 then None 83 83 - else 84 84 - let try_parse prefix relop = 85 85 - if String.starts_with ~prefix s then 86 86 - let v = String.sub s (String.length prefix) (String.length s - String.length prefix) in 87 87 - Some (relop, OpamPackage.Version.of_string v) 88 88 - else None 89 89 - in 90 90 - match try_parse ">=" `Geq with 91 91 - | Some c -> Some c 92 92 - | None -> ( 93 93 - match try_parse "<=" `Leq with 94 94 - | Some c -> Some c 95 95 - | None -> ( 96 96 - match try_parse ">" `Gt with 97 97 - | Some c -> Some c 98 98 - | None -> ( 99 99 - match try_parse "<" `Lt with 100 100 - | Some c -> Some c 101 101 - | None -> ( 102 102 - match try_parse "!=" `Neq with 103 103 - | Some c -> Some c 104 104 - | None -> ( 105 105 - match try_parse "=" `Eq with 106 106 - | Some c -> Some c 107 107 - | None -> 108 108 - (* Treat bare version as exact match *) 109 109 - Some (`Eq, OpamPackage.Version.of_string s)))))) 110 110 - 111 111 - let parse_package_spec s = 112 112 - try 113 113 - let s = String.trim s in 114 114 - (* Check for constraint operators *) 115 115 - let has_constraint = 116 116 - String.contains s '>' || String.contains s '<' 117 117 - || String.contains s '=' || String.contains s '!' 118 118 - in 119 119 - if has_constraint then 120 120 - (* Find where constraint starts *) 121 121 - let constraint_start = 122 122 - let find_op c = try Some (String.index s c) with Not_found -> None in 123 123 - [ find_op '>'; find_op '<'; find_op '='; find_op '!' ] 124 124 - |> List.filter_map Fun.id 125 125 - |> List.fold_left min (String.length s) 126 126 - in 127 127 - let name_part = String.sub s 0 constraint_start in 128 128 - let constraint_part = 129 129 - String.sub s constraint_start (String.length s - constraint_start) 130 130 - in 131 131 - let name = OpamPackage.Name.of_string name_part in 132 132 - let constraint_ = parse_constraint constraint_part in 133 133 - Ok { name; constraint_ } 134 134 - else 135 135 - (* Check for pkg.version format *) 136 136 - match String.rindex_opt s '.' with 137 137 - | Some i when i > 0 -> 138 138 - let name_part = String.sub s 0 i in 139 139 - let version_part = String.sub s (i + 1) (String.length s - i - 1) in 140 140 - (* Validate that version_part looks like a version *) 141 141 - if String.length version_part > 0 142 142 - && (version_part.[0] >= '0' && version_part.[0] <= '9' || version_part.[0] = 'v') 143 143 - then 144 144 - let name = OpamPackage.Name.of_string name_part in 145 145 - let version = OpamPackage.Version.of_string version_part in 146 146 - Ok { name; constraint_ = Some (`Eq, version) } 147 147 - else 148 148 - (* Treat as package name without constraint *) 149 149 - let name = OpamPackage.Name.of_string s in 150 150 - Ok { name; constraint_ = None } 151 151 - | _ -> 152 152 - let name = OpamPackage.Name.of_string s in 153 153 - Ok { name; constraint_ = None } 154 154 - with e -> Error (Printexc.to_string e) 155 155 - 156 156 - let package_spec_to_string spec = 157 157 - let name = OpamPackage.Name.to_string spec.name in 158 158 - match spec.constraint_ with 159 159 - | None -> name 160 160 - | Some (op, v) -> 161 161 - let op_s = 162 162 - match op with 163 163 - | `Eq -> "=" 164 164 - | `Neq -> "!=" 165 165 - | `Geq -> ">=" 166 166 - | `Gt -> ">" 167 167 - | `Leq -> "<=" 168 168 - | `Lt -> "<" 169 169 - in 170 170 - name ^ op_s ^ OpamPackage.Version.to_string v 171 171 - 172 172 - (* Parse compiler spec string like "ocaml.5.4.0" or "5.4.0" *) 173 173 - let parse_compiler_spec (s : string) : package_spec option = 174 174 - let s = String.trim s in 175 175 - if s = "" then None 176 176 - else 177 177 - (* Handle formats: "ocaml.5.4.0", "5.4.0", "ocaml>=5.0" *) 178 178 - let spec_str = 179 179 - if String.starts_with ~prefix:"ocaml" s then s 180 180 - else if s.[0] >= '0' && s.[0] <= '9' then "ocaml." ^ s 181 181 - else s 182 182 - in 183 183 - match parse_package_spec spec_str with 184 184 - | Ok spec -> Some spec 185 185 - | Error _ -> None 186 186 - 187 187 - (* Selection results *) 188 188 - type selection_result = { packages : Repo_index.package_info list } 189 189 - 190 190 - (* Get latest version of each package that is available on the platform *) 191 191 - let latest_versions ?(platform=debian_x86_64) (index : Repo_index.t) : Repo_index.package_info list = 192 192 - let env = make_filter_env platform in 193 193 - let names = Repo_index.package_names index in 194 194 - List.filter_map 195 195 - (fun name -> 196 196 - let versions = Repo_index.find_package name index in 197 197 - (* Filter by availability and sort by version descending *) 198 198 - let available_versions = 199 199 - List.filter (fun (info : Repo_index.package_info) -> 200 200 - is_available_on_platform env info.opam) versions 201 201 - in 202 202 - match 203 203 - List.sort 204 204 - (fun (a : Repo_index.package_info) b -> 205 205 - OpamPackage.Version.compare b.version a.version) 206 206 - available_versions 207 207 - with 208 208 - | latest :: _ -> Some latest 209 209 - | [] -> None) 210 210 - names 211 211 - 212 212 - let select_all index = { packages = latest_versions index } 213 213 - 214 214 - (* Check if a package version satisfies a constraint *) 215 215 - let satisfies_constraint version = function 216 216 - | None -> true 217 217 - | Some (op, cv) -> ( 218 218 - let cmp = OpamPackage.Version.compare version cv in 219 219 - match op with 220 220 - | `Eq -> cmp = 0 221 221 - | `Neq -> cmp <> 0 222 222 - | `Geq -> cmp >= 0 223 223 - | `Gt -> cmp > 0 224 224 - | `Leq -> cmp <= 0 225 225 - | `Lt -> cmp < 0) 1 1 + (** Dependency solver using 0install algorithm. *) 226 2 227 227 - let select_packages ?(platform=debian_x86_64) index specs = 228 228 - if specs = [] then Ok (select_all index) 229 229 - else 230 230 - let env = make_filter_env platform in 231 231 - let selected = 232 232 - List.filter_map 233 233 - (fun spec -> 234 234 - let versions = Repo_index.find_package spec.name index in 235 235 - (* Filter by constraint, availability, and get latest matching *) 236 236 - let matching = 237 237 - List.filter 238 238 - (fun (info : Repo_index.package_info) -> 239 239 - satisfies_constraint info.version spec.constraint_ 240 240 - && is_available_on_platform env info.opam) 241 241 - versions 242 242 - in 243 243 - match 244 244 - List.sort 245 245 - (fun (a : Repo_index.package_info) b -> 246 246 - OpamPackage.Version.compare b.version a.version) 247 247 - matching 248 248 - with 249 249 - | latest :: _ -> Some latest 250 250 - | [] -> None) 251 251 - specs 252 252 - in 253 253 - Ok { packages = selected } 3 3 + let ( / ) = Filename.concat 254 4 255 255 - (* Build a version map for CUDF conversion *) 256 256 - let build_version_map (packages : Repo_index.package_info list) : int OpamPackage.Map.t = 257 257 - (* Group by name and sort versions *) 258 258 - let by_name = Hashtbl.create 256 in 259 259 - List.iter (fun (info : Repo_index.package_info) -> 260 260 - let name = info.name in 261 261 - let versions = try Hashtbl.find by_name name with Not_found -> [] in 262 262 - Hashtbl.replace by_name name (info.version :: versions)) 263 263 - packages; 264 264 - (* Assign version numbers *) 265 265 - let version_map = ref OpamPackage.Map.empty in 266 266 - Hashtbl.iter (fun name versions -> 267 267 - let sorted = List.sort OpamPackage.Version.compare versions in 268 268 - List.iteri (fun i v -> 269 269 - let nv = OpamPackage.create name v in 270 270 - version_map := OpamPackage.Map.add nv (i + 1) !version_map) 271 271 - sorted) 272 272 - by_name; 273 273 - !version_map 5 5 + (** List directory entries, returns empty list if directory doesn't exist. *) 6 6 + let list_dir path = 7 7 + try Sys.readdir path |> Array.to_list 8 8 + with Sys_error _ -> [] 274 9 275 275 - (* Convert opam formula to CUDF vpkgformula (list of disjunctions for AND semantics) 276 276 - Simplified: we ignore version constraints and just require the package exists. 277 277 - This allows the 0install solver to pick the best version. *) 278 278 - let formula_to_vpkgformula available_names (formula : OpamTypes.formula) : Cudf_types.vpkgformula = 279 279 - let atoms = OpamFormula.atoms formula in 280 280 - List.filter_map (fun (name, _version_constraint) -> 281 281 - let name_s = OpamPackage.Name.to_string name in 282 282 - (* Only include dependency if package exists in our available set *) 283 283 - if not (Hashtbl.mem available_names name_s) then None 284 284 - else Some [(name_s, None)]) (* No version constraint - solver picks best *) 285 285 - atoms 10 10 + (** Known compiler packages to filter out. *) 11 11 + let is_compiler_package name = 12 12 + let s = OpamPackage.Name.to_string name in 13 13 + String.starts_with ~prefix:"ocaml-base-compiler" s || 14 14 + String.starts_with ~prefix:"ocaml-variants" s || 15 15 + String.starts_with ~prefix:"ocaml-system" s || 16 16 + String.starts_with ~prefix:"ocaml-config" s || 17 17 + s = "ocaml" || 18 18 + s = "base-unix" || 19 19 + s = "base-threads" || 20 20 + s = "base-bigarray" || 21 21 + s = "base-domains" || 22 22 + s = "base-nnp" 286 23 287 287 - (* For conflicts, we ignore them in CUDF since proper conflict handling requires 288 288 - complex version mapping. The 0install solver will still produce valid results 289 289 - since we filter packages by platform availability. *) 290 290 - let formula_to_vpkglist (_formula : OpamTypes.formula) : Cudf_types.vpkglist = 291 291 - [] (* Ignore conflicts for simplicity *) 24 24 + (** Check if a package has the compiler flag. *) 25 25 + let has_compiler_flag opam = 26 26 + let flags = OpamFile.OPAM.flags opam in 27 27 + List.mem OpamTypes.Pkgflag_Compiler flags 292 28 293 293 - (* Build CUDF universe from packages *) 294 294 - let build_cudf_universe ?(platform=debian_x86_64) (packages : Repo_index.package_info list) = 295 295 - let env = make_filter_env platform in 296 296 - let version_map = build_version_map packages in 29 29 + (** Multi-repo context that searches multiple opam repository directories. *) 30 30 + module Multi_context : sig 31 31 + include Opam_0install.S.CONTEXT 297 32 298 298 - (* First, collect all available package names *) 299 299 - let available_names = Hashtbl.create 256 in 300 300 - List.iter (fun (info : Repo_index.package_info) -> 301 301 - Hashtbl.replace available_names (OpamPackage.Name.to_string info.name) ()) 302 302 - packages; 33 33 + val create : 34 34 + ?constraints:OpamFormula.version_constraint OpamTypes.name_map -> 35 35 + repos:string list -> 36 36 + ocaml_version:string -> 37 37 + unit -> t 38 38 + end = struct 39 39 + type rejection = 40 40 + | UserConstraint of OpamFormula.atom 41 41 + | Unavailable 42 42 + | CompilerPackage 303 43 304 304 - let cudf_packages = List.filter_map (fun (info : Repo_index.package_info) -> 305 305 - let nv = OpamPackage.create info.name info.version in 306 306 - match OpamPackage.Map.find_opt nv version_map with 307 307 - | None -> None 308 308 - | Some cudf_version -> 309 309 - (* Get and filter dependencies *) 310 310 - let depends_formula = OpamFile.OPAM.depends info.opam in 311 311 - let filtered_depends = filter_depends env depends_formula in 312 312 - let depends = formula_to_vpkgformula available_names filtered_depends in 44 44 + let pp_rejection f = function 45 45 + | UserConstraint x -> Fmt.pf f "Rejected by user-specified constraint %s" (OpamFormula.string_of_atom x) 46 46 + | Unavailable -> Fmt.pf f "Availability condition not satisfied" 47 47 + | CompilerPackage -> Fmt.pf f "Compiler package (filtered out)" 313 48 314 314 - (* Get conflicts - simplified to empty for now *) 315 315 - let conflicts_formula = OpamFile.OPAM.conflicts info.opam in 316 316 - let filtered_conflicts = filter_depends env conflicts_formula in 317 317 - let conflicts = formula_to_vpkglist filtered_conflicts in 49 49 + type t = { 50 50 + repos : string list; (* List of packages/ directories *) 51 51 + constraints : OpamFormula.version_constraint OpamTypes.name_map; 52 52 + ocaml_version : string; 53 53 + } 318 54 319 319 - Some { 320 320 - Cudf.default_package with 321 321 - package = OpamPackage.Name.to_string info.name; 322 322 - version = cudf_version; 323 323 - depends = depends; 324 324 - conflicts = conflicts; 325 325 - installed = false; 326 326 - pkg_extra = [ 327 327 - (OpamCudf.s_source, `String (OpamPackage.Name.to_string info.name)); 328 328 - (OpamCudf.s_source_number, `String (OpamPackage.Version.to_string info.version)); 329 329 - ]; 330 330 - }) 331 331 - packages 332 332 - in 55 55 + let env t _pkg v = 56 56 + match OpamVariable.Full.to_string v with 57 57 + | "arch" -> Some (OpamTypes.S "x86_64") 58 58 + | "os" -> Some (OpamTypes.S "linux") 59 59 + | "os-distribution" -> Some (OpamTypes.S "debian") 60 60 + | "os-version" -> Some (OpamTypes.S "12") 61 61 + | "os-family" -> Some (OpamTypes.S "debian") 62 62 + | "opam-version" -> Some (OpamTypes.S "2.2.0") 63 63 + | "sys-ocaml-version" -> Some (OpamTypes.S t.ocaml_version) 64 64 + | "ocaml:native" -> Some (OpamTypes.B true) 65 65 + | _ -> None 333 66 334 334 - let universe = Cudf.load_universe cudf_packages in 335 335 - (universe, version_map) 67 67 + let filter_deps t pkg f = 68 68 + f 69 69 + |> OpamFilter.partial_filter_formula (env t pkg) 70 70 + |> OpamFilter.filter_deps ~build:true ~post:true ~test:false ~doc:false ~dev:false ~dev_setup:false ~default:false 336 71 337 337 - (* Resolve dependencies using 0install solver *) 338 338 - let resolve_deps ?(platform=debian_x86_64) ?compiler index (root_specs : package_spec list) = 339 339 - let env = make_filter_env platform in 72 72 + let user_restrictions t name = 73 73 + OpamPackage.Name.Map.find_opt name t.constraints 340 74 341 341 - (* Get all available packages *) 342 342 - let all_packages = 343 343 - List.filter (fun (info : Repo_index.package_info) -> 344 344 - is_available_on_platform env info.opam) 345 345 - (Repo_index.all_packages index) 346 346 - in 75 75 + (** Load opam file from path. *) 76 76 + let load_opam path = 77 77 + try Some (OpamFile.OPAM.read (OpamFile.make (OpamFilename.raw path))) 78 78 + with _ -> None 347 79 348 348 - (* Build CUDF universe *) 349 349 - let universe, version_map = build_cudf_universe ~platform all_packages in 80 80 + (** Create a minimal virtual opam file for base packages. *) 81 81 + let virtual_opam () = 82 82 + OpamFile.OPAM.empty 350 83 351 351 - (* Build request - add compiler if specified *) 352 352 - let all_specs = match compiler with 353 353 - | Some compiler_spec -> compiler_spec :: root_specs 354 354 - | None -> root_specs 355 355 - in 84 84 + (** Find all versions of a package across all repos. *) 85 85 + let find_versions t name = 86 86 + let name_str = OpamPackage.Name.to_string name in 87 87 + (* Collect versions from all repos, first repo wins for duplicates *) 88 88 + let seen = Hashtbl.create 16 in 89 89 + List.iter (fun packages_dir -> 90 90 + let pkg_dir = packages_dir / name_str in 91 91 + list_dir pkg_dir |> List.iter (fun entry -> 92 92 + match OpamPackage.of_string_opt entry with 93 93 + | Some pkg when OpamPackage.name pkg = name -> 94 94 + let v = OpamPackage.version pkg in 95 95 + if not (Hashtbl.mem seen v) then begin 96 96 + let opam_path = pkg_dir / entry / "opam" in 97 97 + Hashtbl.add seen v opam_path 98 98 + end 99 99 + | _ -> () 100 100 + ) 101 101 + ) t.repos; 102 102 + Hashtbl.fold (fun v path acc -> (v, path) :: acc) seen [] 356 103 357 357 - let requested = List.filter_map (fun spec -> 358 358 - let name_s = OpamPackage.Name.to_string spec.name in 359 359 - (* Check if package exists in universe *) 360 360 - if Cudf.mem_package universe (name_s, 1) || 361 361 - List.exists (fun p -> p.Cudf.package = name_s) (Cudf.get_packages universe) 362 362 - then Some (name_s, `Essential) 363 363 - else begin 364 364 - Format.eprintf "Warning: Package %s not found in universe@." name_s; 365 365 - None 366 366 - end) 367 367 - all_specs 368 368 - in 104 104 + let candidates t name = 105 105 + let name_str = OpamPackage.Name.to_string name in 106 106 + (* Provide virtual packages for compiler/base packages at the configured version *) 107 107 + if name_str = "ocaml" then 108 108 + let v = OpamPackage.Version.of_string t.ocaml_version in 109 109 + [v, Ok (virtual_opam ())] 110 110 + else if name_str = "base-unix" || name_str = "base-threads" || 111 111 + name_str = "base-bigarray" || name_str = "base-domains" || 112 112 + name_str = "base-nnp" then 113 113 + let v = OpamPackage.Version.of_string "base" in 114 114 + [v, Ok (virtual_opam ())] 115 115 + else if is_compiler_package name then 116 116 + (* Other compiler packages - not available *) 117 117 + [] 118 118 + else 119 119 + let user_constraints = user_restrictions t name in 120 120 + find_versions t name 121 121 + |> List.sort (fun (v1, _) (v2, _) -> OpamPackage.Version.compare v2 v1) (* Prefer newest *) 122 122 + |> List.map (fun (v, opam_path) -> 123 123 + match user_constraints with 124 124 + | Some test when not (OpamFormula.check_version_formula (OpamFormula.Atom test) v) -> 125 125 + v, Error (UserConstraint (name, Some test)) 126 126 + | _ -> 127 127 + match load_opam opam_path with 128 128 + | None -> v, Error Unavailable 129 129 + | Some opam -> 130 130 + (* Check flags:compiler *) 131 131 + if has_compiler_flag opam then 132 132 + v, Error CompilerPackage 133 133 + else 134 134 + (* Check available filter *) 135 135 + let pkg = OpamPackage.create name v in 136 136 + let available = OpamFile.OPAM.available opam in 137 137 + match OpamFilter.eval ~default:(OpamTypes.B false) (env t pkg) available with 138 138 + | B true -> v, Ok opam 139 139 + | _ -> v, Error Unavailable 140 140 + ) 369 141 370 370 - if requested = [] then 371 371 - Error "No valid packages to resolve" 372 372 - else 373 373 - (* Create solver and solve *) 374 374 - let solver = Opam_0install_cudf.create ~constraints:[] universe in 375 375 - match Opam_0install_cudf.solve solver requested with 376 376 - | Error diag -> 377 377 - Error (Opam_0install_cudf.diagnostics diag) 378 378 - | Ok selections -> 379 379 - (* Convert results back to package info *) 380 380 - let selected_cudf = Opam_0install_cudf.packages_of_result selections in 381 381 - let selected_packages = List.filter_map (fun (name, cudf_version) -> 382 382 - (* Find the opam package *) 383 383 - let opam_name = OpamPackage.Name.of_string name in 384 384 - let versions = Repo_index.find_package opam_name index in 385 385 - (* Get the version that matches *) 386 386 - List.find_opt (fun (info : Repo_index.package_info) -> 387 387 - let nv = OpamPackage.create info.name info.version in 388 388 - match OpamPackage.Map.find_opt nv version_map with 389 389 - | Some v -> v = cudf_version 390 390 - | None -> false) 391 391 - versions) 392 392 - selected_cudf 393 393 - in 394 394 - (* Deduplicate by package name+version *) 395 395 - let seen = Hashtbl.create 64 in 396 396 - let unique_packages = List.filter (fun (info : Repo_index.package_info) -> 397 397 - let key = OpamPackage.to_string (OpamPackage.create info.name info.version) in 398 398 - if Hashtbl.mem seen key then false 399 399 - else begin Hashtbl.add seen key (); true end) 400 400 - selected_packages 401 401 - in 402 402 - (* Filter out compiler packages from results *) 403 403 - let non_compiler_packages = List.filter (fun (info : Repo_index.package_info) -> 404 404 - not (is_compiler_package info.opam info.name)) 405 405 - unique_packages 406 406 - in 407 407 - Ok { packages = non_compiler_packages } 142 142 + let create ?(constraints=OpamPackage.Name.Map.empty) ~repos ~ocaml_version () = 143 143 + (* Convert repo roots to packages/ directories *) 144 144 + let packages_dirs = List.map (fun r -> r / "packages") repos in 145 145 + { repos = packages_dirs; constraints; ocaml_version } 146 146 + end 408 147 409 409 - let select_with_deps ?(platform=debian_x86_64) ?compiler index specs = 410 410 - if specs = [] then Ok (select_all index) 411 411 - else 412 412 - resolve_deps ~platform ?compiler index specs 148 148 + module Solver = Opam_0install.Solver.Make(Multi_context) 413 149 414 414 - (* Cmdliner integration *) 150 150 + type solve_result = { 151 151 + packages : OpamPackage.t list; 152 152 + } 415 153 416 416 - let package_specs_conv : package_spec Arg.conv = 417 417 - let parse s = 418 418 - match parse_package_spec s with 419 419 - | Ok spec -> Ok spec 420 420 - | Error msg -> Error (`Msg msg) 421 421 - in 422 422 - let print fmt spec = Format.pp_print_string fmt (package_spec_to_string spec) in 423 423 - Arg.conv (parse, print) 154 154 + type solve_error = string 424 155 425 425 - let package_specs_term : package_spec list Term.t = 426 426 - let doc = 427 427 - "Package specification. Can be a package name (any version), \ 428 428 - name.version (exact version), or name>=version (constraint). \ 429 429 - Examples: cmdliner, lwt.5.6.0, dune>=3.0" 430 430 - in 431 431 - Arg.(value & pos_all package_specs_conv [] & info [] ~docv:"PACKAGE" ~doc) 156 156 + (** Solve dependencies for a list of package names. *) 157 157 + let solve ~repos ~ocaml_version ~packages : (solve_result, solve_error) result = 158 158 + let context = Multi_context.create ~repos ~ocaml_version () in 159 159 + let names = List.map OpamPackage.Name.of_string packages in 160 160 + match Solver.solve context names with 161 161 + | Ok selections -> 162 162 + let pkgs = Solver.packages_of_result selections in 163 163 + (* Filter out compiler packages from result *) 164 164 + let pkgs = List.filter (fun pkg -> 165 165 + not (is_compiler_package (OpamPackage.name pkg)) 166 166 + ) pkgs in 167 167 + Ok { packages = pkgs } 168 168 + | Error diagnostics -> 169 169 + Error (Solver.diagnostics diagnostics)

+24 -79

lib/opam/solver.mli

··· 1 1 - (** Package selection with constraint solving. 1 1 + (** Dependency solver using 0install algorithm. 2 2 3 3 - Uses the 0install solver (via opam-0install-cudf) to select 4 4 - a consistent set of packages based on constraints, filtered 5 5 - for Debian x86_64 platform. *) 3 3 + Solves package dependencies across multiple configured opam repositories, 4 4 + filtering out compiler packages and respecting availability constraints. *) 6 5 7 7 - (** {1 Platform Configuration} *) 8 8 - 9 9 - type platform = { 10 10 - os : string; 11 11 - os_family : string; 12 12 - os_distribution : string; 13 13 - arch : string; 6 6 + type solve_result = { 7 7 + packages : OpamPackage.t list; 8 8 + (** List of packages that need to be installed, including transitive deps. *) 14 9 } 15 15 - (** Target platform for filtering packages. *) 16 10 17 17 - val debian_x86_64 : platform 18 18 - (** Default platform: Debian Linux on x86_64. *) 11 11 + type solve_error = string 12 12 + (** Human-readable error message explaining why solving failed. *) 19 13 20 20 - (** {1 Package Specifications} *) 14 14 + val solve : 15 15 + repos:string list -> 16 16 + ocaml_version:string -> 17 17 + packages:string list -> 18 18 + (solve_result, solve_error) result 19 19 + (** [solve ~repos ~ocaml_version ~packages] solves dependencies for [packages]. 21 20 22 22 - type version_constraint = OpamFormula.relop * OpamPackage.Version.t 23 23 - (** A version constraint like [>=, 1.0.0]. *) 21 21 + @param repos List of opam repository root directories (containing packages/) 22 22 + @param ocaml_version The OCaml compiler version to solve for (e.g. "5.2.0") 23 23 + @param packages List of package names to solve for 24 24 25 25 - type package_spec = { 26 26 - name : OpamPackage.Name.t; 27 27 - constraint_ : version_constraint option; 28 28 - } 29 29 - (** A package specification with optional version constraint. *) 25 25 + Returns the full list of packages (including transitive dependencies) that 26 26 + need to be installed, or an error message if solving failed. 30 27 31 31 - val parse_package_spec : string -> (package_spec, string) result 32 32 - (** [parse_package_spec s] parses a package spec string like: 33 33 - - "pkg" (any version) 34 34 - - "pkg.1.0.0" (exact version) 35 35 - - "pkg>=1.0.0" (version constraint) 36 36 - - "pkg<2.0" (version constraint) *) 28 28 + Compiler packages (ocaml-base-compiler, base-*, etc.) are automatically 29 29 + filtered out since they are assumed to be pre-installed. *) 37 30 38 38 - val package_spec_to_string : package_spec -> string 39 39 - (** [package_spec_to_string spec] converts a spec back to string form. *) 40 40 - 41 41 - val parse_compiler_spec : string -> package_spec option 42 42 - (** [parse_compiler_spec s] parses a compiler version string like: 43 43 - - "5.4.0" (parsed as ocaml.5.4.0) 44 44 - - "ocaml.5.4.0" (exact version) 45 45 - - "ocaml>=5.0" (version constraint) 46 46 - Returns None if the string is empty or invalid. *) 47 47 - 48 48 - (** {1 Selection} *) 49 49 - 50 50 - type selection_result = { 51 51 - packages : Repo_index.package_info list; 52 52 - (** Selected packages that satisfy all constraints. *) 53 53 - } 54 54 - (** Result of package selection. *) 55 55 - 56 56 - val select_all : Repo_index.t -> selection_result 57 57 - (** [select_all index] returns all packages (latest version of each) 58 58 - that are available on the target platform (Debian x86_64). *) 59 59 - 60 60 - val select_packages : 61 61 - ?platform:platform -> 62 62 - Repo_index.t -> package_spec list -> (selection_result, string) result 63 63 - (** [select_packages index specs] finds packages matching the given 64 64 - specifications, filtered by platform availability. Returns the 65 65 - latest compatible version of each package. *) 66 66 - 67 67 - val select_with_deps : 68 68 - ?platform:platform -> 69 69 - ?compiler:package_spec -> 70 70 - Repo_index.t -> package_spec list -> (selection_result, string) result 71 71 - (** [select_with_deps ?platform ?compiler index specs] selects packages and 72 72 - their transitive dependencies using the 0install solver. 73 73 - 74 74 - - Dependencies are filtered by platform (Debian x86_64 by default) 75 75 - - If [compiler] is specified, it is added as a constraint and all 76 76 - compiler-related packages (those with flags:compiler or matching 77 77 - known compiler package patterns) are filtered from the results 78 78 - - The solver finds a consistent installation set *) 79 79 - 80 80 - (** {1 Cmdliner Integration} *) 81 81 - 82 82 - val package_specs_term : package_spec list Cmdliner.Term.t 83 83 - (** Cmdliner term for parsing package specifications from command line. 84 84 - Accepts zero or more package specs as positional arguments. 85 85 - If no packages specified, returns empty list (meaning "all packages"). *) 86 86 - 87 87 - val package_specs_conv : package_spec Cmdliner.Arg.conv 88 88 - (** Cmdliner converter for a single package spec. *) 31 31 + val is_compiler_package : OpamPackage.Name.t -> bool 32 32 + (** [is_compiler_package name] returns true if [name] is a known compiler 33 33 + or base package that should be filtered out. *)

-237

lib/opam/source.ml

··· 1 1 - type source_kind = Archive | Git 2 2 - 3 3 - type archive_source = { 4 4 - url : string; 5 5 - checksums : string list; 6 6 - mirrors : string list; 7 7 - } 8 8 - 9 9 - type git_source = { 10 10 - url : string; 11 11 - branch : string option; 12 12 - } 13 13 - 14 14 - type source = ArchiveSource of archive_source | GitSource of git_source | NoSource 15 15 - 16 16 - type package_source = { 17 17 - name : string; 18 18 - version : string; 19 19 - source : source; 20 20 - dev_repo : Dev_repo.t option; 21 21 - } 22 22 - 23 23 - type grouped_sources = { 24 24 - dev_repo : Dev_repo.t option; 25 25 - packages : package_source list; 26 26 - } 27 27 - 28 28 - (* Helper to check if URL is git-like *) 29 29 - let is_git_url url = 30 30 - let s = OpamUrl.to_string url in 31 31 - String.starts_with ~prefix:"git" s 32 32 - || String.ends_with ~suffix:".git" s 33 33 - || url.OpamUrl.backend = `git 34 34 - 35 35 - (* Extract archive source from opam URL.t *) 36 36 - let extract_archive_from_url (url_t : OpamFile.URL.t) : archive_source = 37 37 - let main_url = OpamFile.URL.url url_t in 38 38 - let checksums = 39 39 - OpamFile.URL.checksum url_t 40 40 - |> List.map OpamHash.to_string 41 41 - in 42 42 - let mirrors = 43 43 - OpamFile.URL.mirrors url_t 44 44 - |> List.map OpamUrl.to_string 45 45 - in 46 46 - { url = OpamUrl.to_string main_url; checksums; mirrors } 47 47 - 48 48 - (* Extract git source from OpamUrl.t *) 49 49 - let normalize_git_url s = 50 50 - (* Strip git+ prefix so URLs work directly with git clone *) 51 51 - if String.starts_with ~prefix:"git+" s then 52 52 - String.sub s 4 (String.length s - 4) 53 53 - else s 54 54 - 55 55 - let extract_git_from_url (url : OpamUrl.t) : git_source = 56 56 - { url = normalize_git_url (OpamUrl.to_string url); branch = url.OpamUrl.hash } 57 57 - 58 58 - let extract kind (info : Repo_index.package_info) : package_source = 59 59 - let name = OpamPackage.Name.to_string info.name in 60 60 - let version = OpamPackage.Version.to_string info.version in 61 61 - let dev_repo = info.dev_repo in 62 62 - let source = 63 63 - match kind with 64 64 - | Archive -> ( 65 65 - match OpamFile.OPAM.url info.opam with 66 66 - | Some url_t -> ArchiveSource (extract_archive_from_url url_t) 67 67 - | None -> NoSource) 68 68 - | Git -> ( 69 69 - (* Prefer dev-repo for git, fall back to url if it's a git URL *) 70 70 - match OpamFile.OPAM.dev_repo info.opam with 71 71 - | Some url -> GitSource (extract_git_from_url url) 72 72 - | None -> ( 73 73 - match OpamFile.OPAM.url info.opam with 74 74 - | Some url_t -> 75 75 - let main_url = OpamFile.URL.url url_t in 76 76 - if is_git_url main_url then 77 77 - GitSource (extract_git_from_url main_url) 78 78 - else NoSource 79 79 - | None -> NoSource)) 80 80 - in 81 81 - { name; version; source; dev_repo } 82 82 - 83 83 - let extract_all kind packages = List.map (extract kind) packages 84 84 - 85 85 - let group_by_dev_repo (sources : package_source list) : grouped_sources list = 86 86 - (* Build separate lists: one map for packages with dev_repo, one list for those without *) 87 87 - let with_repo, without_repo = 88 88 - List.partition (fun (src : package_source) -> Option.is_some src.dev_repo) sources 89 89 - in 90 90 - (* Group packages with dev_repo *) 91 91 - let add_to_map map (src : package_source) = 92 92 - match src.dev_repo with 93 93 - | Some dr -> 94 94 - let existing = 95 95 - match Dev_repo.Map.find_opt dr map with 96 96 - | Some l -> l 97 97 - | None -> [] 98 98 - in 99 99 - Dev_repo.Map.add dr (src :: existing) map 100 100 - | None -> map 101 101 - in 102 102 - let map = List.fold_left add_to_map Dev_repo.Map.empty with_repo in 103 103 - (* Convert map to list of grouped_sources *) 104 104 - let grouped_with_repo = 105 105 - Dev_repo.Map.fold 106 106 - (fun dr pkgs acc -> 107 107 - let packages = List.rev pkgs in (* Preserve original order *) 108 108 - { dev_repo = Some dr; packages } :: acc) 109 109 - map [] 110 110 - |> List.sort (fun a b -> 111 111 - match (a.dev_repo, b.dev_repo) with 112 112 - | Some a, Some b -> Dev_repo.compare a b 113 113 - | _ -> 0) 114 114 - in 115 115 - (* Add packages without dev_repo at the end *) 116 116 - if without_repo = [] then grouped_with_repo 117 117 - else grouped_with_repo @ [{ dev_repo = None; packages = without_repo }] 118 118 - 119 119 - (* JSON Codecs - simplified with tagged object *) 120 120 - 121 121 - let source_jsont : source Jsont.t = 122 122 - let open Jsont in 123 123 - Object.map ~kind:"source" 124 124 - (fun source_type url checksums mirrors branch -> 125 125 - match source_type with 126 126 - | "archive" -> 127 127 - let checksums = match checksums with Some cs -> cs | None -> [] in 128 128 - ArchiveSource { url; checksums; mirrors } 129 129 - | "git" -> GitSource { url; branch } 130 130 - | _ -> NoSource) 131 131 - |> Object.mem "type" string ~enc:(function 132 132 - | ArchiveSource _ -> "archive" 133 133 - | GitSource _ -> "git" 134 134 - | NoSource -> "none") 135 135 - |> Object.mem "url" string ~dec_absent:"" ~enc:(function 136 136 - | ArchiveSource a -> a.url 137 137 - | GitSource g -> g.url 138 138 - | NoSource -> "") 139 139 - |> Object.opt_mem "checksums" (list string) ~enc:(function 140 140 - | ArchiveSource a -> Some a.checksums 141 141 - | _ -> None) 142 142 - |> Object.mem "mirrors" (list string) ~dec_absent:[] ~enc:(function 143 143 - | ArchiveSource a -> a.mirrors 144 144 - | _ -> []) 145 145 - |> Object.opt_mem "branch" string ~enc:(function 146 146 - | GitSource g -> g.branch 147 147 - | _ -> None) 148 148 - |> Object.finish 149 149 - 150 150 - let dev_repo_jsont = 151 151 - Jsont.( 152 152 - map 153 153 - ~dec:(fun s -> Dev_repo.of_string s) 154 154 - ~enc:Dev_repo.to_string string) 155 155 - 156 156 - let package_source_jsont : package_source Jsont.t = 157 157 - let open Jsont in 158 158 - Object.map ~kind:"package_source" 159 159 - (fun name version source dev_repo -> 160 160 - ({ name; version; source; dev_repo } : package_source)) 161 161 - |> Object.mem "name" string ~enc:(fun (p : package_source) -> p.name) 162 162 - |> Object.mem "version" string ~enc:(fun (p : package_source) -> p.version) 163 163 - |> Object.mem "source" source_jsont ~enc:(fun (p : package_source) -> p.source) 164 164 - |> Object.opt_mem "dev_repo" dev_repo_jsont ~enc:(fun (p : package_source) -> p.dev_repo) 165 165 - |> Object.finish 166 166 - 167 167 - let package_sources_jsont = Jsont.list package_source_jsont 168 168 - 169 169 - let grouped_sources_jsont : grouped_sources Jsont.t = 170 170 - let open Jsont in 171 171 - Object.map ~kind:"grouped_sources" 172 172 - (fun dev_repo packages -> ({ dev_repo; packages } : grouped_sources)) 173 173 - |> Object.opt_mem "dev_repo" dev_repo_jsont ~enc:(fun (g : grouped_sources) -> g.dev_repo) 174 174 - |> Object.mem "packages" (list package_source_jsont) ~enc:(fun (g : grouped_sources) -> g.packages) 175 175 - |> Object.finish 176 176 - 177 177 - let grouped_sources_list_jsont = Jsont.list grouped_sources_jsont 178 178 - 179 179 - (* TOML Codecs *) 180 180 - 181 181 - let source_tomlt : source Tomlt.t = 182 182 - let open Tomlt in 183 183 - let open Table in 184 184 - obj (fun source_type url checksums mirrors branch -> 185 185 - match source_type with 186 186 - | "archive" -> 187 187 - let checksums = match checksums with Some cs -> cs | None -> [] in 188 188 - ArchiveSource { url; checksums; mirrors } 189 189 - | "git" -> GitSource { url; branch } 190 190 - | "none" | _ -> NoSource) 191 191 - |> mem "type" string ~enc:(function 192 192 - | ArchiveSource _ -> "archive" 193 193 - | GitSource _ -> "git" 194 194 - | NoSource -> "none") 195 195 - |> mem "url" string ~dec_absent:"" ~enc:(function 196 196 - | ArchiveSource a -> a.url 197 197 - | GitSource g -> g.url 198 198 - | NoSource -> "") 199 199 - |> opt_mem "checksums" (list string) ~enc:(function 200 200 - | ArchiveSource a -> Some a.checksums 201 201 - | _ -> None) 202 202 - |> mem "mirrors" (list string) ~dec_absent:[] ~enc:(function 203 203 - | ArchiveSource a -> a.mirrors 204 204 - | _ -> []) 205 205 - |> opt_mem "branch" string ~enc:(function 206 206 - | GitSource g -> g.branch 207 207 - | _ -> None) 208 208 - |> finish 209 209 - 210 210 - let dev_repo_tomlt = 211 211 - Tomlt.( 212 212 - map 213 213 - ~dec:(fun s -> Dev_repo.of_string s) 214 214 - ~enc:Dev_repo.to_string string) 215 215 - 216 216 - let package_source_tomlt : package_source Tomlt.t = 217 217 - let open Tomlt in 218 218 - let open Table in 219 219 - obj (fun name version source dev_repo -> 220 220 - ({ name; version; source; dev_repo } : package_source)) 221 221 - |> mem "name" string ~enc:(fun (p : package_source) -> p.name) 222 222 - |> mem "version" string ~enc:(fun (p : package_source) -> p.version) 223 223 - |> mem "source" source_tomlt ~enc:(fun (p : package_source) -> p.source) 224 224 - |> opt_mem "dev_repo" dev_repo_tomlt ~enc:(fun (p : package_source) -> p.dev_repo) 225 225 - |> finish 226 226 - 227 227 - let package_sources_tomlt = Tomlt.array_of_tables package_source_tomlt 228 228 - 229 229 - let grouped_sources_tomlt : grouped_sources Tomlt.t = 230 230 - let open Tomlt in 231 231 - let open Table in 232 232 - obj (fun dev_repo packages -> ({ dev_repo; packages } : grouped_sources)) 233 233 - |> opt_mem "dev_repo" dev_repo_tomlt ~enc:(fun (g : grouped_sources) -> g.dev_repo) 234 234 - |> mem "packages" (array_of_tables package_source_tomlt) ~enc:(fun (g : grouped_sources) -> g.packages) 235 235 - |> finish 236 236 - 237 237 - let grouped_sources_list_tomlt = Tomlt.array_of_tables grouped_sources_tomlt

-85

lib/opam/source.mli

··· 1 1 - (** Package source URL extraction. 2 2 - 3 3 - Extracts download URLs or git remotes from opam package metadata. *) 4 4 - 5 5 - (** {1 Source Types} *) 6 6 - 7 7 - type source_kind = 8 8 - | Archive (** Tarball/archive URL with optional checksums *) 9 9 - | Git (** Git repository URL *) 10 10 - (** The kind of source to extract. *) 11 11 - 12 12 - type archive_source = { 13 13 - url : string; 14 14 - checksums : string list; (** SHA256, MD5, etc. *) 15 15 - mirrors : string list; 16 16 - } 17 17 - (** An archive source with URL and integrity info. *) 18 18 - 19 19 - type git_source = { 20 20 - url : string; 21 21 - branch : string option; (** Branch/tag/ref if specified *) 22 22 - } 23 23 - (** A git repository source. *) 24 24 - 25 25 - type source = 26 26 - | ArchiveSource of archive_source 27 27 - | GitSource of git_source 28 28 - | NoSource 29 29 - (** A package source. *) 30 30 - 31 31 - type package_source = { 32 32 - name : string; 33 33 - version : string; 34 34 - source : source; 35 35 - dev_repo : Dev_repo.t option; 36 36 - } 37 37 - (** A package with its source and dev-repo for grouping. *) 38 38 - 39 39 - type grouped_sources = { 40 40 - dev_repo : Dev_repo.t option; 41 41 - packages : package_source list; 42 42 - } 43 43 - (** Packages grouped by their shared dev-repo. *) 44 44 - 45 45 - (** {1 Extraction} *) 46 46 - 47 47 - val extract : source_kind -> Repo_index.package_info -> package_source 48 48 - (** [extract kind info] extracts the source of the specified kind from 49 49 - package [info]. For [Archive], uses the url field. For [Git], uses 50 50 - dev-repo or falls back to url if it's a git URL. *) 51 51 - 52 52 - val extract_all : source_kind -> Repo_index.package_info list -> package_source list 53 53 - (** [extract_all kind packages] extracts sources for all packages. *) 54 54 - 55 55 - val group_by_dev_repo : package_source list -> grouped_sources list 56 56 - (** [group_by_dev_repo sources] groups packages by their dev-repo. 57 57 - Packages with the same dev-repo are grouped together since they 58 58 - come from the same repository. Groups with dev-repo are sorted first, 59 59 - followed by packages without dev-repo. *) 60 60 - 61 61 - (** {1 Codecs} *) 62 62 - 63 63 - val package_source_jsont : package_source Jsont.t 64 64 - (** JSON codec for a package source. *) 65 65 - 66 66 - val package_sources_jsont : package_source list Jsont.t 67 67 - (** JSON codec for a list of package sources. *) 68 68 - 69 69 - val grouped_sources_jsont : grouped_sources Jsont.t 70 70 - (** JSON codec for grouped sources. *) 71 71 - 72 72 - val grouped_sources_list_jsont : grouped_sources list Jsont.t 73 73 - (** JSON codec for a list of grouped sources. *) 74 74 - 75 75 - val package_source_tomlt : package_source Tomlt.t 76 76 - (** TOML codec for a package source. *) 77 77 - 78 78 - val package_sources_tomlt : package_source list Tomlt.t 79 79 - (** TOML codec for a list of package sources (as array of tables). *) 80 80 - 81 81 - val grouped_sources_tomlt : grouped_sources Tomlt.t 82 82 - (** TOML codec for grouped sources. *) 83 83 - 84 84 - val grouped_sources_list_tomlt : grouped_sources list Tomlt.t 85 85 - (** TOML codec for a list of grouped sources (as array of tables). *)

+1

lib/unpac.ml

··· 6 6 module Git_repo_lookup = Git_repo_lookup 7 7 module Worktree = Worktree 8 8 module Config = Config 9 9 + module Vendor_cache = Vendor_cache 9 10 module Init = Init 10 11 module Backend = Backend

+157

lib/vendor_cache.ml

··· 1 1 + (** Vendor cache - a persistent bare git repository for caching upstream fetches. 2 2 + 3 3 + The cache stores fetched repositories as remotes/branches, allowing multiple 4 4 + unpac projects to share fetched content without re-downloading. *) 5 5 + 6 6 + (** {1 Types} *) 7 7 + 8 8 + type t = Eio.Fs.dir_ty Eio.Path.t 9 9 + (** Path to the cache bare repository *) 10 10 + 11 11 + (** {1 Cache Location} *) 12 12 + 13 13 + let default_path () = 14 14 + let cache_home = 15 15 + match Sys.getenv_opt "XDG_CACHE_HOME" with 16 16 + | Some dir -> dir 17 17 + | None -> 18 18 + match Sys.getenv_opt "HOME" with 19 19 + | Some home -> Filename.concat home ".cache" 20 20 + | None -> "/tmp" 21 21 + in 22 22 + Filename.concat cache_home "unpac/vendor-cache" 23 23 + 24 24 + (** {1 Initialization} *) 25 25 + 26 26 + let init ~proc_mgr ~fs ?path () = 27 27 + let cache_path = match path with 28 28 + | Some p -> p 29 29 + | None -> default_path () 30 30 + in 31 31 + let cache = Eio.Path.(fs / cache_path) in 32 32 + 33 33 + (* Check if already initialized *) 34 34 + if Eio.Path.is_directory cache then 35 35 + cache 36 36 + else begin 37 37 + (* Create parent directories *) 38 38 + let parent = Filename.dirname cache_path in 39 39 + let parent_path = Eio.Path.(fs / parent) in 40 40 + Eio.Path.mkdirs ~exists_ok:true ~perm:0o755 parent_path; 41 41 + 42 42 + (* Initialize bare repository *) 43 43 + Eio.Path.mkdirs ~exists_ok:false ~perm:0o755 cache; 44 44 + Git.run_exn ~proc_mgr ~cwd:cache ["init"; "--bare"] |> ignore; 45 45 + cache 46 46 + end 47 47 + 48 48 + (** {1 Remote Naming} 49 49 + 50 50 + We use URL-based remote names to avoid conflicts. 51 51 + e.g., "github.com/dbuenzli/astring" for https://github.com/dbuenzli/astring.git *) 52 52 + 53 53 + let url_to_remote_name url = 54 54 + (* Strip protocol and .git suffix *) 55 55 + let url = 56 56 + let prefixes = ["https://"; "http://"; "git://"; "ssh://"; "git@"] in 57 57 + List.fold_left (fun u prefix -> 58 58 + if String.starts_with ~prefix u then 59 59 + String.sub u (String.length prefix) (String.length u - String.length prefix) 60 60 + else u 61 61 + ) url prefixes 62 62 + in 63 63 + let url = 64 64 + if String.ends_with ~suffix:".git" url then 65 65 + String.sub url 0 (String.length url - 4) 66 66 + else url 67 67 + in 68 68 + (* Replace : with / for git@ style URLs *) 69 69 + String.map (fun c -> if c = ':' then '/' else c) url 70 70 + 71 71 + let branch_name ~remote ~branch = 72 72 + remote ^ "/" ^ branch 73 73 + 74 74 + (** {1 Cache Operations} *) 75 75 + 76 76 + let has_remote ~proc_mgr cache remote_name = 77 77 + match Git.remote_url ~proc_mgr ~cwd:cache remote_name with 78 78 + | Some _ -> true 79 79 + | None -> false 80 80 + 81 81 + let ensure_remote ~proc_mgr cache ~url = 82 82 + let remote_name = url_to_remote_name url in 83 83 + if has_remote ~proc_mgr cache remote_name then 84 84 + remote_name 85 85 + else begin 86 86 + Git.run_exn ~proc_mgr ~cwd:cache 87 87 + ["remote"; "add"; remote_name; url] |> ignore; 88 88 + remote_name 89 89 + end 90 90 + 91 91 + let fetch ~proc_mgr cache ~url = 92 92 + let remote_name = ensure_remote ~proc_mgr cache ~url in 93 93 + Git.fetch ~proc_mgr ~cwd:cache ~remote:remote_name; 94 94 + remote_name 95 95 + 96 96 + let get_ref ~proc_mgr cache ~url ~branch = 97 97 + let remote_name = url_to_remote_name url in 98 98 + let ref_name = branch_name ~remote:remote_name ~branch in 99 99 + match Git.rev_parse ~proc_mgr ~cwd:cache ref_name with 100 100 + | Some sha -> Some sha 101 101 + | None -> None 102 102 + 103 103 + (** Fetch to cache, then clone ref into project's bare repo *) 104 104 + let fetch_to_project ~proc_mgr ~cache ~project_git ~url ~branch = 105 105 + (* First, fetch to cache (include tags, force update to avoid conflicts) *) 106 106 + let remote_name = ensure_remote ~proc_mgr cache ~url in 107 107 + Git.run_exn ~proc_mgr ~cwd:cache 108 108 + ["fetch"; "--tags"; "--force"; remote_name] |> ignore; 109 109 + 110 110 + (* Determine if this is a branch or tag *) 111 111 + let branch_ref = branch_name ~remote:remote_name ~branch in 112 112 + let tag_ref = "refs/tags/" ^ branch in 113 113 + 114 114 + (* Check which ref exists in cache *) 115 115 + let cache_ref = 116 116 + match Git.rev_parse ~proc_mgr ~cwd:cache branch_ref with 117 117 + | Some _ -> branch_ref 118 118 + | None -> 119 119 + (* Try as a tag *) 120 120 + match Git.rev_parse ~proc_mgr ~cwd:cache tag_ref with 121 121 + | Some _ -> tag_ref 122 122 + | None -> failwith (Printf.sprintf "Ref not found: %s (tried branch %s and tag %s)" 123 123 + branch branch_ref tag_ref) 124 124 + in 125 125 + 126 126 + (* Now fetch from cache into project *) 127 127 + let cache_path = snd cache in 128 128 + 129 129 + (* Add cache as a remote in project if not exists *) 130 130 + let cache_remote = "vendor-cache" in 131 131 + (match Git.remote_url ~proc_mgr ~cwd:project_git cache_remote with 132 132 + | None -> 133 133 + Git.run_exn ~proc_mgr ~cwd:project_git 134 134 + ["remote"; "add"; cache_remote; cache_path] |> ignore 135 135 + | Some _ -> ()); 136 136 + 137 137 + (* Fetch the specific ref from cache *) 138 138 + Git.run_exn ~proc_mgr ~cwd:project_git 139 139 + ["fetch"; cache_remote; cache_ref ^ ":" ^ cache_ref] |> ignore; 140 140 + 141 141 + cache_ref 142 142 + 143 143 + (** {1 Listing} *) 144 144 + 145 145 + let list_remotes ~proc_mgr cache = 146 146 + Git.run_lines ~proc_mgr ~cwd:cache ["remote"] 147 147 + 148 148 + let list_branches ~proc_mgr cache = 149 149 + Git.run_lines ~proc_mgr ~cwd:cache ["branch"; "-a"] 150 150 + |> List.filter_map (fun line -> 151 151 + let line = String.trim line in 152 152 + if String.starts_with ~prefix:"* " line then 153 153 + Some (String.sub line 2 (String.length line - 2)) 154 154 + else if line <> "" then 155 155 + Some line 156 156 + else 157 157 + None)

+63

lib/vendor_cache.mli

··· 1 1 + (** Vendor cache - a persistent bare git repository for caching upstream fetches. 2 2 + 3 3 + The cache stores fetched repositories as remotes/branches, allowing multiple 4 4 + unpac projects to share fetched content without re-downloading. *) 5 5 + 6 6 + (** {1 Types} *) 7 7 + 8 8 + type t = Eio.Fs.dir_ty Eio.Path.t 9 9 + (** Path to the cache bare repository *) 10 10 + 11 11 + (** {1 Cache Location} *) 12 12 + 13 13 + val default_path : unit -> string 14 14 + (** Returns the default cache path (XDG_CACHE_HOME/unpac/vendor-cache) *) 15 15 + 16 16 + (** {1 Initialization} *) 17 17 + 18 18 + val init : proc_mgr:Git.proc_mgr -> fs:Eio.Fs.dir_ty Eio.Path.t -> ?path:string -> unit -> t 19 19 + (** [init ~proc_mgr ~fs ?path ()] initializes and returns the cache. 20 20 + Creates the bare repository if it doesn't exist. 21 21 + @param path Optional custom cache path. Uses default if not provided. *) 22 22 + 23 23 + (** {1 Remote Naming} *) 24 24 + 25 25 + val url_to_remote_name : string -> string 26 26 + (** [url_to_remote_name url] converts a git URL to a remote name. 27 27 + e.g., "https://github.com/dbuenzli/astring.git" -> "github.com/dbuenzli/astring" *) 28 28 + 29 29 + val branch_name : remote:string -> branch:string -> string 30 30 + (** [branch_name ~remote ~branch] returns the full branch name in cache. *) 31 31 + 32 32 + (** {1 Cache Operations} *) 33 33 + 34 34 + val has_remote : proc_mgr:Git.proc_mgr -> t -> string -> bool 35 35 + (** [has_remote ~proc_mgr cache name] checks if a remote exists in cache. *) 36 36 + 37 37 + val ensure_remote : proc_mgr:Git.proc_mgr -> t -> url:string -> string 38 38 + (** [ensure_remote ~proc_mgr cache ~url] adds remote if needed, returns remote name. *) 39 39 + 40 40 + val fetch : proc_mgr:Git.proc_mgr -> t -> url:string -> string 41 41 + (** [fetch ~proc_mgr cache ~url] fetches from URL into cache, returns remote name. *) 42 42 + 43 43 + val get_ref : proc_mgr:Git.proc_mgr -> t -> url:string -> branch:string -> string option 44 44 + (** [get_ref ~proc_mgr cache ~url ~branch] returns the SHA for a cached ref. *) 45 45 + 46 46 + val fetch_to_project : 47 47 + proc_mgr:Git.proc_mgr -> 48 48 + cache:t -> 49 49 + project_git:Eio.Fs.dir_ty Eio.Path.t -> 50 50 + url:string -> 51 51 + branch:string -> 52 52 + string 53 53 + (** [fetch_to_project ~proc_mgr ~cache ~project_git ~url ~branch] 54 54 + fetches from upstream to cache, then from cache to project's bare repo. 55 55 + Returns the cache ref name. *) 56 56 + 57 57 + (** {1 Listing} *) 58 58 + 59 59 + val list_remotes : proc_mgr:Git.proc_mgr -> t -> string list 60 60 + (** List all remotes in the cache. *) 61 61 + 62 62 + val list_branches : proc_mgr:Git.proc_mgr -> t -> string list 63 63 + (** List all branches in the cache. *)

+2

test/cram/dune

··· 1 1 + (cram 2 2 + (deps %{bin:unpac}))

+147

test/cram/full-workflow.t

··· 1 1 + Full workflow test: init -> repo -> add by name -> edit -> merge 2 2 + 3 3 + Setup: create a test opam repository with a package 4 4 + 5 5 + $ REPO_ROOT=$(pwd) 6 6 + $ mkdir -p test-opam-repo/packages/testlib/testlib.1.0.0 7 7 + $ mkdir -p test-pkg 8 8 + $ cd test-pkg 9 9 + $ git init -q 10 10 + $ git config user.email "test@test.com" 11 11 + $ git config user.name "Test" 12 12 + $ cat > lib.ml << 'EOF' 13 13 + > let greet name = "Hello, " ^ name 14 14 + > EOF 15 15 + $ git add -A 16 16 + $ git commit -q -m "Initial commit" 17 17 + $ TEST_PKG_PATH=$(pwd) 18 18 + $ cd "$REPO_ROOT" 19 19 + 20 20 + Create opam file pointing to test-pkg 21 21 + 22 22 + $ cat > test-opam-repo/packages/testlib/testlib.1.0.0/opam << EOF 23 23 + > opam-version: "2.0" 24 24 + > synopsis: "A test library" 25 25 + > dev-repo: "git+file://$TEST_PKG_PATH" 26 26 + > EOF 27 27 + 28 28 + Initialize unpac project 29 29 + 30 30 + $ unpac init myproj 31 31 + Initialized unpac project at myproj 32 32 + 33 33 + Next steps: 34 34 + cd myproj 35 35 + unpac opam repo add <name> <path> # configure opam repository 36 36 + unpac project new <name> # create a project 37 37 + $ cd myproj 38 38 + 39 39 + Configure opam repository 40 40 + 41 41 + $ unpac opam repo list 42 42 + No repositories configured 43 43 + 44 44 + Hint: unpac opam repo add <name> <path> 45 45 + 46 46 + $ unpac opam repo add myrepo "$REPO_ROOT/test-opam-repo" 47 47 + Added repository myrepo at $TESTCASE_ROOT/test-opam-repo 48 48 + 49 49 + Next: unpac opam add <package> # vendor a package by name 50 50 + 51 51 + $ unpac opam repo list 52 52 + myrepo: $TESTCASE_ROOT/test-opam-repo 53 53 + 54 54 + Create a project 55 55 + 56 56 + $ unpac project new myapp 57 57 + Created project myapp 58 58 + 59 59 + Next steps: 60 60 + unpac opam add <package> # vendor a package 61 61 + unpac opam merge <package> myapp # merge package into project 62 62 + 63 63 + Add package by name (looks up in configured repo) 64 64 + 65 65 + $ unpac opam add testlib 2>&1 | grep -oE "^(Added|Error) \w+" 66 66 + Added testlib 67 67 + 68 68 + Verify package was added 69 69 + 70 70 + $ unpac opam list 71 71 + testlib 72 72 + 73 73 + Check package info 74 74 + 75 75 + $ unpac opam info testlib | grep -E "^(Package|Local)" 76 76 + Package: testlib 77 77 + Local commits: 0 78 78 + 79 79 + No local changes yet 80 80 + 81 81 + $ unpac opam diff testlib 82 82 + No local changes 83 83 + 84 84 + Hint: unpac opam edit testlib # to make changes 85 85 + 86 86 + Edit the package 87 87 + 88 88 + $ unpac opam edit testlib 2>&1 | head -1 89 89 + Editing testlib 90 90 + 91 91 + Make a change in the patches worktree 92 92 + 93 93 + $ echo 'let goodbye () = "Goodbye!"' >> opam/patches/testlib/vendor/opam/testlib/lib.ml 94 94 + $ (cd opam/patches/testlib && git add -A && git commit -q -m "Add goodbye function") 95 95 + 96 96 + Now we have local changes 97 97 + 98 98 + $ unpac opam info testlib | grep "Local commits" 99 99 + Local commits: 1 100 100 + 101 101 + $ unpac opam diff testlib | grep -E "^\+" | tail -1 102 102 + +let goodbye () = "Goodbye!" 103 103 + 104 104 + Close editing session 105 105 + 106 106 + $ unpac opam done testlib 107 107 + Closed editing session for testlib 108 108 + 109 109 + Next steps: 110 110 + unpac opam diff testlib # view your changes 111 111 + unpac opam merge testlib <project> # merge into a project 112 112 + 113 113 + Merge into project 114 114 + 115 115 + $ unpac opam merge testlib myapp 2>&1 | grep -E "^(Merged|Merge)" 116 116 + Merged testlib into project myapp 117 117 + 118 118 + Verify files in project 119 119 + 120 120 + $ ls project/myapp/vendor/opam/testlib 121 121 + lib.ml 122 122 + 123 123 + $ cat project/myapp/vendor/opam/testlib/lib.ml 124 124 + let greet name = "Hello, " ^ name 125 125 + let goodbye () = "Goodbye!" 126 126 + 127 127 + Remove package 128 128 + 129 129 + $ unpac opam remove testlib 130 130 + Removed testlib 131 131 + 132 132 + Hint: unpac opam add <package> # to add another package 133 133 + 134 134 + $ unpac opam list 135 135 + No packages vendored 136 136 + 137 137 + Hint: unpac opam add <package> 138 138 + 139 139 + Remove repository 140 140 + 141 141 + $ unpac opam repo remove myrepo 142 142 + Removed repository myrepo 143 143 + 144 144 + $ unpac opam repo list 145 145 + No repositories configured 146 146 + 147 147 + Hint: unpac opam repo add <name> <path>

+37

test/cram/init.t

··· 1 1 + Initialize a new unpac project 2 2 + 3 3 + $ unpac init myproject 4 4 + Initialized unpac project at myproject 5 5 + 6 6 + Next steps: 7 7 + cd myproject 8 8 + unpac opam repo add <name> <path> # configure opam repository 9 9 + unpac project new <name> # create a project 10 10 + 11 11 + Check that the directory structure was created 12 12 + 13 13 + $ ls myproject 14 14 + git 15 15 + main 16 16 + 17 17 + Check git directory contents (bare repo) 18 18 + 19 19 + $ ls myproject/git | grep -E "^(HEAD|config|objects|refs|worktrees)$" 20 20 + HEAD 21 21 + config 22 22 + objects 23 23 + refs 24 24 + worktrees 25 25 + 26 26 + $ ls myproject/main 27 27 + unpac.toml 28 28 + 29 29 + Check git branches (main is in worktree so shows +) 30 30 + 31 31 + $ git -C myproject/git branch 32 32 + + main 33 33 + 34 34 + Init should fail if directory exists 35 35 + 36 36 + $ unpac init myproject 2>&1 | grep "uncaught exception" | head -1 37 37 + unpac: internal error, uncaught exception:

+86

test/cram/opam.t

··· 1 1 + Test opam vendoring commands 2 2 + 3 3 + Setup: create a local test package repo 4 4 + 5 5 + $ mkdir -p test-pkg 6 6 + $ cd test-pkg 7 7 + $ git init -q 8 8 + $ git config user.email "test@test.com" 9 9 + $ git config user.name "Test" 10 10 + $ cat > dune-project << 'EOF' 11 11 + > (lang dune 3.0) 12 12 + > (name testpkg) 13 13 + > EOF 14 14 + $ cat > lib.ml << 'EOF' 15 15 + > let hello () = "Hello from testpkg" 16 16 + > EOF 17 17 + $ git add -A 18 18 + $ git commit -q -m "Initial commit" 19 19 + $ cd .. 20 20 + 21 21 + Create unpac project 22 22 + 23 23 + $ unpac init myproj 24 24 + Initialized unpac project at myproj 25 25 + 26 26 + Next steps: 27 27 + cd myproj 28 28 + unpac opam repo add <name> <path> # configure opam repository 29 29 + unpac project new <name> # create a project 30 30 + $ cd myproj 31 31 + 32 32 + Initially no packages 33 33 + 34 34 + $ unpac opam list 35 35 + No packages vendored 36 36 + 37 37 + Hint: unpac opam add <package> 38 38 + 39 39 + Add the test package (use absolute path for local git repos) 40 40 + 41 41 + $ unpac opam add "$(realpath ../test-pkg)" --name testpkg 2>&1 | grep -oE "^(Added|Error) \w+" 42 42 + Added testpkg 43 43 + 44 44 + List shows the package 45 45 + 46 46 + $ unpac opam list 47 47 + testpkg 48 48 + 49 49 + Check branches were created 50 50 + 51 51 + $ git -C git branch | grep opam | sort 52 52 + opam/patches/testpkg 53 53 + opam/upstream/testpkg 54 54 + opam/vendor/testpkg 55 55 + 56 56 + Check vendor branch has prefixed content 57 57 + 58 58 + $ git -C git show opam/vendor/testpkg --name-only | grep "^vendor/" 59 59 + vendor/opam/testpkg/dune-project 60 60 + vendor/opam/testpkg/lib.ml 61 61 + 62 62 + Create a project and merge 63 63 + 64 64 + $ unpac project new myapp 65 65 + Created project myapp 66 66 + 67 67 + Next steps: 68 68 + unpac opam add <package> # vendor a package 69 69 + unpac opam merge <package> myapp # merge package into project 70 70 + 71 71 + $ unpac opam merge testpkg myapp 2>&1 | grep -E "^(Merged|Merge conflict)" 72 72 + Merged testpkg into project myapp 73 73 + 74 74 + Check files appear in project 75 75 + 76 76 + $ ls project/myapp/vendor/opam/testpkg 77 77 + dune-project 78 78 + lib.ml 79 79 + 80 80 + $ cat project/myapp/vendor/opam/testpkg/lib.ml 81 81 + let hello () = "Hello from testpkg" 82 82 + 83 83 + Check git log shows merge 84 84 + 85 85 + $ git -C project/myapp log --oneline | wc -l 86 86 + 3

+71

test/cram/project.t

··· 1 1 + Test project management commands 2 2 + 3 3 + Setup: create an unpac project first 4 4 + 5 5 + $ unpac init testproj 6 6 + Initialized unpac project at testproj 7 7 + 8 8 + Next steps: 9 9 + cd testproj 10 10 + unpac opam repo add <name> <path> # configure opam repository 11 11 + unpac project new <name> # create a project 12 12 + $ cd testproj 13 13 + 14 14 + Create a new project 15 15 + 16 16 + $ unpac project new myapp 17 17 + Created project myapp 18 18 + 19 19 + Next steps: 20 20 + unpac opam add <package> # vendor a package 21 21 + unpac opam merge <package> myapp # merge package into project 22 22 + 23 23 + Check project directory was created 24 24 + 25 25 + $ ls project/myapp 26 26 + dune 27 27 + dune-project 28 28 + vendor 29 29 + 30 30 + Check template files 31 31 + 32 32 + $ cat project/myapp/dune-project 33 33 + (lang dune 3.20) 34 34 + (name myapp) 35 35 + 36 36 + $ cat project/myapp/dune 37 37 + (vendored_dirs vendor) 38 38 + 39 39 + Check vendor directory structure 40 40 + 41 41 + $ ls project/myapp/vendor 42 42 + opam 43 43 + 44 44 + List projects 45 45 + 46 46 + $ unpac project list 47 47 + myapp 48 48 + 49 49 + Check git branches (filter out main, show project branches) 50 50 + 51 51 + $ git -C git branch | grep project 52 52 + + project/myapp 53 53 + 54 54 + Create another project 55 55 + 56 56 + $ unpac project new otherapp 57 57 + Created project otherapp 58 58 + 59 59 + Next steps: 60 60 + unpac opam add <package> # vendor a package 61 61 + unpac opam merge <package> otherapp # merge package into project 62 62 + 63 63 + $ unpac project list 64 64 + myapp 65 65 + otherapp 66 66 + 67 67 + Projects should appear in unpac.toml 68 68 + 69 69 + $ grep -E "^(myapp|otherapp)" main/unpac.toml 70 70 + myapp = {} 71 71 + otherapp = {}

+2 -4

unpac-opam.opam

··· 10 10 "unpac" 11 11 "opam-format" 12 12 "opam-core" 13 13 - "opam-repository" 14 14 - "opam-solver" 15 15 - "opam-0install-cudf" 16 16 - "cudf" 13 13 + "opam-state" 14 14 + "opam-0install" 17 15 "cmdliner" {>= "1.2.0"} 18 16 "odoc" {with-doc} 19 17 ]