VarName rework (#150)

Co-authored-by: Xianda Sun <5433119+sunxd3@users.noreply.github.com>

Author: penelopeysm

HISTORY.md

@@ -1,3 +1,56 @@
+## 0.14.0
+
+This release overhauls the `VarName` type.
+Much of the external API for traversing and manipulating `VarName`s (once they have been constructed) has been preserved, but if you use the `VarName` type directly, there are significant changes.
+
+**Internal representation**
+
+The `optic` field of VarName now uses our hand-rolled optic types, which are subtypes of `AbstractPPL.AbstractOptic`.
+Previously these were optics from Accessors.jl.
+
+This change was made for two reasons: firstly, it is easier to provide custom behaviour for VarNames as we avoid running into possible type piracy issues, and secondly, the linked-list data structure used in `AbstractOptic` is easier to work with than Accessors.jl, which used `Base.ComposedFunction` to represent optic compositions and required a lot of care to avoid a litany of issues with associativity and identity optics (see e.g. https://github.com/JuliaLang/julia/pull/54877).
+
+To construct an optic, the easiest way is to use the `@opticof` macro, which superficially behaves similarly to `Accessors.@optic` (for example, you can write `@opticof _[1].y.z`), but also supports automatic concretization by passing a second parameter (just like `@varname`).
+
+**Concretization**
+
+VarNames using 'dynamic' indices, i.e., `begin` and `end`, are now instantiated in a 'dynamic' form, meaning that these indices are unresolved.
+These indices need to be resolved, or concretized, against the actual container.
+For example, `@varname(x[end])` is dynamic, but when concretized against `x = randn(3)`, this becomes `@varname(x[3])`.
+This can be done using `concretize(varname, x)`.
+
+The idea of concretization is not new to AbstractPPL.
+However, there are some differences:
+
+  - Colons are no longer concretized: they *always* remain as Colons, even after calling `concretize`.
+  - Previously, AbstractPPL would refuse to allow you to construct unconcretized versions of `begin` and `end`. This is no longer the case; you can now create such VarNames in their unconcretized forms.
+    This is useful, for example, when indexing into a chain that contains `x` as a variable-length vector. This change allows you to write `chain[@varname(x[end])]` without having AbstractPPL throw an error.
+
+**Keyword arguments to `getindex`**
+
+VarNames can now be constructed with keyword arguments in `Index` optics, for example `@varname(x[i=1])`.
+This is specifically implemented to support DimensionalData.jl's DimArrays.
+
+**Other interface functions**
+
+The `vsym` function (and `@vsym`) has been removed; you should use `getsym(vn)` instead.
+
+The `Base.get` and `Accessors.set` methods for VarNames have been removed (these were responsible for method ambiguities).
+Instead of using these methods you can first convert the `VarName` to an optic using `varname_to_optic(vn)`, and then use the getter and setter methods on the optics.
+
+VarNames cannot be composed with optics now (compose the optics yourself).
+
+The `inspace` function has been removed.
+It used to be relevant for Turing's old Gibbs sampler; but now it no longer serves any use.
+
+`ConcretizedSlice` has been removed (since colons are no longer concretized).
+
+The subsumption interface has been pared down to just a single function, `subsumes`.
+All other functions, such as `subsumedby`, `uncomparable`, and the Unicode operators, have been removed.
+
+Serialization still works exactly as before.
+However, you will see differences in the serialization output compared to previous versions, due to the changes in the internal structure.
+
 ## 0.13.6
 
 Fix a missing qualifier in AbstractPPLDistributionsExt.

Project.toml

@@ -3,30 +3,36 @@ uuid = "7a57a42e-76ec-4ea3-a279-07e840d6d9cf"
 keywords = ["probablistic programming"]
 license = "MIT"
 desc = "Common interfaces for probabilistic programming"
-version = "0.13.6"
+version = "0.14.0"
 
 [deps]
 AbstractMCMC = "80f14c24-f653-4e6a-9b94-39d6b0f70001"
 Accessors = "7d9f7c33-5ae7-4f3b-8dc6-eff91059b697"
+BangBang = "198e06fe-97b7-11e9-32a5-e1d131e6ad66"
 DensityInterface = "b429d917-457f-4dbc-8f4c-0cc954292b1d"
 JSON = "682c06a0-de6a-54ab-a142-c8b1cf79cde6"
 LinearAlgebra = "37e2e46d-f89d-539d-b4ee-838fcccc9c8e"
+MacroTools = "1914dd2f-81c6-5fcd-8719-6d5c9610ff09"
+OrderedCollections = "bac558e1-5e72-5ebc-8fee-abe8a469f55d"
 Random = "9a3f8284-a2c9-5f02-9a11-845980a1fd5c"
 StatsBase = "2913bbd2-ae8a-5f71-8c99-4fb6c76f3a91"
 
 [weakdeps]
 Distributions = "31c24e10-a181-5473-b8eb-7969acd0382f"
 
 [extensions]
-AbstractPPLDistributionsExt = ["Distributions"]
+AbstractPPLDistributionsExt = ["Distributions", "LinearAlgebra"]
 
 [compat]
 AbstractMCMC = "2, 3, 4, 5"
 Accessors = "0.1"
+BangBang = "0.4"
 DensityInterface = "0.4"
 Distributions = "0.25"
 JSON = "0.19 - 0.21, 1"
-LinearAlgebra = "<0.0.1, 1.10"
+LinearAlgebra = "<0.0.1, 1"
+MacroTools = "0.5"
+OrderedCollections = "1.8.1"
 Random = "1.6"
 StatsBase = "0.32, 0.33, 0.34"
-julia = "1.10"
+julia = "1.10.8"

docs/Project.toml

@@ -3,3 +3,6 @@ AbstractPPL = "7a57a42e-76ec-4ea3-a279-07e840d6d9cf"
 Distributions = "31c24e10-a181-5473-b8eb-7969acd0382f"
 Documenter = "e30172f5-a6a5-5a46-863b-614d45cd2de4"
 LinearAlgebra = "37e2e46d-f89d-539d-b4ee-838fcccc9c8e"
+
+[sources]
+AbstractPPL = {path = "../"}

docs/make.jl

@@ -9,7 +9,7 @@ DocMeta.setdocmeta!(AbstractPPL, :DocTestSetup, :(using AbstractPPL); recursive=
 makedocs(;
     sitename="AbstractPPL",
     modules=[AbstractPPL, Base.get_extension(AbstractPPL, :AbstractPPLDistributionsExt)],
-    pages=["index.md", "api.md", "interface.md"],
+    pages=["index.md", "varname.md", "pplapi.md", "interface.md"],
     checkdocs=:exports,
     doctest=false,
 )

docs/src/api.md

@@ -0,0 +1,20 @@
+# Probabilistic programming API
+
+## Abstract model functions
+
+```@docs
+AbstractProbabilisticProgram
+condition
+decondition
+fix
+unfix
+logdensityof
+AbstractContext
+evaluate!!
+```
+
+## Abstract traces
+
+```@docs
+AbstractModelTrace
+```