All very LLM driven by me so far but hopefully can grow it into something. There are a few example approaches linked/in dev by me and a few from others linked out to. This was all part of a failed series of team grants so now having a think about what to do.
3 Likes
I have been driving a lot of tokens into this - costing Anthropic a lot of money. Neither approach is quite there (that being said the ComposableTuringIDModels.jl does some pretty trendy stuff now). I am quite excited about the future roadmap for the Distributions.jl stack - starting to get a lot of things for free (see i.e. convolve_series rolling across packages to give hassle free strata/time-varying nested delay convolutions for those that sail those waters).
There is also LoweredDistributions.jl which is a package I wanted @slmontes and @sdwfrost to make for a long time. My version of it is currently I would say totally mad but I think with enough sacrifices to the token gods we may be getting somewhere.
I am finding having a no opt out hard set of package checks pretty critical (Here EpiAwareADTools.jl - Julia has some stuff for this like i.e. BestieTemplate.jl but in general I have found them a bit basic and not sufficiently opinionated (LLMS need to be broken on the wheel of automated AD compatibility testing to get them to write good AD stable Julia code I have found).
If anyone has a token hook up we could drive this faster and further - there is a good case for doing the Google forecasting LLM code writing thing but with packages like this I think - could make some AI nerds look pretty cool and also not be just an advertising gimmick.
(also yes driving this much LLM is making me go mad see: https://samabbott.co.uk/how-I-llm/#/give-the-agents-their-own-name)
2 Likes
@sbfnk has also been doing this cool (but currently unrelated) branching process mega package which has some similar composable ideas running through it:
@slmontes and @sdwfrost also have Contacts.jl (it actually isn’t called this I am just hoping to get it renamed…) which looks lovely. Hopefully going to setup an integration with ComposableTuringIDModels.jl once I land between infection strata mixing (tokens being sacrified with this in mind as we speak). GitHub - epirecipes/ContACT.jl: Category theory applied to contact matrices · GitHub
The idea of the EpiAware ecosystem is that others can contribute packages to it but only if they sign up to be governed (by the community-agreed) iron rules of EpiAwarePackageTool) and have some attemp at interoperability (i.e enforcing within community reverse deps checks). I am hoping to have a community agent that maybe could be grant funded to run org level maintaince tasks (would require some day actually getting some funding…)
1 Like
Ah also the state of some parts of the Julia ecosystem due to lack of maintaince/consistency is really shocking. I think there is a lot of scope for this to be turned around with some more automation and users complaining (i.e. us my impression is its a lot of casual devs who would like to hear their work was being used for pandemic prep work). Of particular note `SpecialFunctions.jl` has no maintainer and works poorly with nearly all AD backends - this is needed for i.e using a Gamma distribution. A bunch of inference packages try to get around this by writing their own distribution packages. Still, then they just become a walled garden of the sort we see in other ecosystems, and any actual benefit of Julia is lost.
Going to Juliacon next week and planning on raving at people over my 3 talks and 3 posters!
I am mostly very keen about all this so I can stop telling people they have done discretisation wrong hit a weekly record of 4 papers this week. For 3 of those it will change the results and i.e. means a rewrite is really in order.
1 Like
I haven’t really directly linked to packages. There are a few that are useful today and are somewhat stable.
Stable & ready to use
- CensoredDistributions: Censored-event distributions for Julia’s `Distributions.jl` (censored/truncated normal, delays, etc.). Headline feature:
convolve_series— series convolution for continuous and time-varying censored delays. - ConvolvedDistributions: Composable convolutions of distributions, including discrete/timed support. The engine behind `convolve_series`.
- ReparameterisedDistributions: Switch parameterisations between distributions cleanly.
- ComposableTuringIDModels: Composable Turing.jl infectious-disease models — multi-strata infection processes, coupling, arbitrary strata mappings. This is the only ID specific package at the moment.
Usable core, more integration to come
- ComposedDistributions: Compose distributions from parts (Sequential, Parallel, Compete, Pool, Varying, …). Mature and well-documented on its own; the bigger payoff comes once it plugs into censored/event handling for discrete events — that integration is still in prototype.
- ModifiedDistributions: Modify distribution behaviour (affine, transformed, weighted forms). Stable ish and documented; most value comes via the same discrete/event integration that’s not yet landed.
- LoweredDistributions: Lowered representations of distributions. Workable, but still a review-before-reliance package — recommend a human check before betting on it.
Prototype
- DistributionsInference: Inference over distribution parameters. Early-stage; API isn’t settled. Watch, don’t adopt yet.