And he's gonna tell us why you're… Uh, inspiring him to do simulation, so yeah.
Right. Thank you. Yeah, so, uh, this is a high-energy theory seminar, so I thought I'd try and make a little particle physics theme, so I'll tell you why neutrino particles inspired me to simulate and reconstruct the entire universe.
So, let me rewind the clock a few years.
Just a few, uh, when I was an undergrad in Imperial College London, in a particle physics lab.
And I asked my advisor. How accurately can we measure the neutrino mass?
And so what he told me was, there's basically two limits that we have.
from neutrino oscillations, you have this lower limit of 0.06 eV, because if neutrinos oscillate, they must have this mass.
And then there's also, uh, beta decay, where neutrons turn into protons, electrons, and neutrinos.
And by measuring the energy of the electron. Uh, you can… the maximum energy of the electron tells you how heavy the neutrino must be, and so it gives you this upper limit of 0.8 eV.
And it turns out that with Cosmology, you can actually bridge this gap.
Um, how do you do that? Here's a picture of the universe from the CMB until today. Light from the CMB travels to us.
Um, and it's been observed by, for example, Plank, and in Planck 2018, it pushed down this upper limit to 0.2 EV.
But then neutrinos don't just impact the CMB. They also changed the way matter clusters. So, in the universe, you have cold dark matter, which is what's responsible for large-scale structure and formation of galaxies and everything, but neutrinos, they have a much, much lighter mass, they have much larger velocities.
And so they don't cluster the same way that Cold Dark Matter does. And so, what it means is if neutrinos have higher mass, if they're responsible for a higher proportion of the matter content in the universe.
You'll have less clustering, and so if you combine CMB data with large-scale structured data, for example, BOS.
BAO, uh, you get even tighter constraints, and so you get this .12e.
Uh, limit when you combine Planck with… with both.
But more recently, Desi has, uh, also measured the VAO and got it even tighter upper bound now, and now we're kind of hitting the slower limit. It's not 5 Sigma, so nothing to get too excited about yet, but there's papers claiming negative neutrino mass, and so it's a very exciting time to be working.
On massive neutrinos with cosmological data. Um, and so, in response to this question, my advisor told me.
Perhaps I should switch to cosmology. This isn't really what he said, but uh… but anyway, so now I do cosmology. And by the way, uh, so my plan for this talk is kind of give a big picture of some of the things that I work on.
Uh, so if you have any questions, you want me to explain anything in more detail, please let me know.
Let me go ahead and just interrupt me. So, cosmology, it's very exciting, we've got all these surveys now, uh, coming out, measuring large-scale structure on the top.
And also, measuring the CMB. And so later in my talk, I'll talk about combining these two datasets together, and the things we can do that. But to start, let me just talk about large-scale structure. How much information is there in the large-scale structure?
Uh, in terms of neutrino mass. And so, the first thing we look at in cosmology is the power spectrum, which is just the furrow transform with a two-point correlation function.
And we know that within linear theory, you can make this plot of the power spectrum as a function of k, and as you increase the neutrino mass, you get the suppression on small scales at high k.
And this is just what I explained earlier. The heavier the neutrinos, the more matter you have moving very quickly, so less clustering on small scales.
Um, and one thing I want to point out already is the BAO. So, when I spoke about BAO earlier and what DESI has been measuring in its, like, flagship analysis so far, is these wiggles in the power spectrum.
Um, and so, just by measuring, say, this first peak, you can learn a lot about cosmology.
And, uh, one interesting thing to note. is that VAO indirectly measures the neutrino mass. So, when DESI reports, uh, postrains or neutrino mass, essentially what's happening.
Is, uh, you get this… well, so the CMV gives you this orange contour.
in this neutrino mass HNO plane. The BAO just measures distances, essentially, so you get this pink contour, which is just constraining H.
And so what's really happening is it's breaking this degeneracy between neutrino mass and distances.
And giving you this black contour in the end. So that's what's giving Desi the tight constraints on neutrino mats.
And so, if we want to do a better job, one thing we could do is try and measure the BAO even better, get an even tighter pink contour, and that will force us to get an even slightly. Another thing you can do, which we'll speak about later, is just when you combine CMB with large-scale structure, go beyond the BAO. Don't just use the BAO.
Uh… Can I ask a question?
Suppose there was some other… fairly relativistic species. Yeah. How would that affect all of these analyses? Sure, yeah, I mean, it would also have effects on the small-scale power spectrum if it was completely degenerate with the effect that massive neutrinos have, then.
I guess it would be hard to disentangle, but I suppose, uh… Yeah, they would maybe have different math scales, or different properties, which you would have to include when you do this kind of modeling.
What are you using about the neutrino properties? Just their maps, or… Yeah. The fact that they don't interact.
Uh, just MS. I mean, based… they still interrupt, uh, yeah, there's no, like, weak or voices or anything in there. It's just the… well, at least in the analyses that we do nowadays, it's purely gravitational, essentially. Yeah.
you're not sensitive to the time at which they decouple.
Um, in these analyses? Uh, yeah, that's all built into the simulation. Like, that goes into… How you, uh… as in, you're not specifically probing that, but the time that they decoupled will.
Play a part in the… But if you shifted that temperature by a little bit, given that it is such a high temperature for these scales, you're probably not going to notice a difference. Yeah, you won't notice it at this level. Yeah, other than… other than, like.
The temperature ratio of the neutrinos. I mean, how do you… how do you determine how many of them there are?
Yeah, that's another… so, you're only sensitive to the total sum of the masses with this kind of analysis.
There are some people who've written papers of ways you could potentially unpack the individual.
Uh, masses, but you'd have to really resolve things very.
We're on very small scales in order to do that. So yeah, right now, Cosmology, and I think over the next few years, is mainly going to be focusing on some of the neutrino masses.
Okay. So, uh… But another thing to keep in mind is… Small scales are non-linear. So this, I'm floating here is the linear power spectrum, but in reality, all these scales, which stage 4 surveys, like Desi, are going to start probing, are very, very small, and.
If you want to model these accurately, you've got to understand how large-scale structure works in the nonlinear regime.
And so, to do that, you need simulations. And his example of some simulations in action. And one thing to keep in mind is.
So, you have delta nu, the neutrino density, you have delta C, the cold dark matter density, and then you add them together to get the total amount of density.
And so, you can see, like, delta C, the cold dark matter, it's really, really clustered. It's very, very sharp points, whereas delta nu is much more coarse, uh, much more smooth, because neutrinos don't clusters much.
And then you add them together to get your total matter field in the end.
And so, to start, let's have a look at how much information there is just in delta M. So if I could measure delta M.
the total matter field, what can I say about neutrino maps?
So, in order to talk about how much information the cosmic field adds, you can start with the power spectrum of the two-point information, which I was just showing earlier, and this is traditionally what cosmologists have done.
But as you go through smaller scales, this is no longer optimal. There's a lot more information in the image, in the map.
Uh, the higher order statistics could give you. And so, one thing you can do is define these higher order summary statistics of your data, and two that I looked at is the halo mass function. So you look at all the really overdense regions in your map, you look at how heavy they are, you make a histogram of that, and use that as a summary. You can also look at something called the void size function, where you look at empty regions in your map, so, like, these green circles.
See how large they are, make a histogram of that.
And then, if you combine all 3 of these.
Uh, in a neutrino mass sigma 8 plane, they have completely different degeneracies, and then when you combine them all together, you get this really tight.
black contour in the middle. So what this is saying is in the delta M field.
There's a lot of information beyond the power spectrum, and you can measure neutrino mass really well if you can measure the total matter field.
Yeah. This is not a… These are all probing things beyond linear order.
Yeah, this is… so we're using nonlinear simulations of the matter field. Is that because of the length scale, I guess for the halo, that's definitely a nonlinear wind scale?
Yeah, so that's an interesting thing. People say void's, uh, quite linear, but, uh, it's… the reason why they're very sensitive to neutrino mass is, like, um.
neutrinos, like, in voids, you tend to have more neutrinos than you have cold dark matter, so it's breaking some kind of degeneracy there, because neutrinos free stream through these under-dense regions, whereas you don't find much cold matter.
That's one of the arguments people get for bullets.
Yeah, so, uh, you just use the two-point function, I guess the theory is fairly robust or well-developed, and you don't need to resort to detailed simulations, I guess, to… At the nonlinear order, no. We still need simulation. No, I know, but the BAO plot, the one you showed previously… Yeah.
That one… so that is, uh, it's still a simulation, but it's, like, equations, differential equations and everything, so very simple, I guess. Yeah, yeah, that's… yeah, that's the solar problem, the linear theory. Okay, and um… In this case, though, you ran a particular simulation, I guess, to get the non-linear order.
Yeah. And then this is the result you got with that particular simulation? Sure, yeah. Okay, so then… How do you show people that the simulation robot? Sure, so I'll speak about that a little bit more, actually, later, but in this case, we're looking at delta M.
Which I think also is a solved problem. So, like, if you run an in-body simulation just with matter particles in there, um.
you know, up to some scale, all the simulations we have have converged, essentially. So I think this we can trust well. When you go to, like, the delta M of, like, basically dark matter halos?
Uh, Delta M being the matter field of neutrino particles, CDM particles, yeah, there's very little that can go wrong there, but then when you go to galaxies and put in all these other kind of things, they're less serious out of distribution. Actually, later in my talk, I'll also mention how.
different simulations, DIFA in that delta M, but the key point, I guess, is I've only gone to K max of 0.5, so I've.
chosen a cutoff scale, at which point I know my simulation has converged.
So this is a pretty good theoretical result. Yeah. What is the vertical axis?
Uh, vertical. The neutrino mass, this. Yes. Uh, so this is the mass of the total sum of the neutrino masses in electron volts.
So, this is because… Sure, yeah, no, I mean, this is a great question. But yeah, really, we've put no prior on here, we're just doing a Fisher analysis in this. So we've got, like, some central value of the neutrino mass, which is.
0, and then we add, like, a little… Uh… like… We have simulations with 0.1 EV neutrino mass, 0.2 EV, 0.4 EV.
Sure, yeah. It's just the… it's just the quirk of the Fisher analysis. All it's really saying is if my neutrino mass is, like, around zero, this is the size of the error bar. That's how I read this plot.
But in reality, you'd put a prior, and this would all be cut off. But this is just a Fisher analysis.
And by Fisher analysis… sorry, what do you mean that you… you have something to take the second derivative of? We have a bunch of simulations out of fixed cosmology. We calculate a covariance matrix. We then have a bunch of simulations which have, like, little perturbations to each parameter, so, like, zero EV neutrino mass and 0.1 eV, say, you've calculated derivative between the output of those two.
And then you just make a definition matrix with the derivatives and the covariates in the middle.
And then it gives you, kind of, the most optimistic information content available for those parameters. This is kind of… in cosmology, the first thing you ever do… It tells you how to draw an ellipse, and then you just grab the ellipse. Yeah, this is how… Without thinking about what the negative actually means.
Uh, well, but in reality, you just cut that off, essentially.
Just saying that as you… if you imagine there's a knob, you're turning from the mass, or the sum of the masses.
Your simulation will change in a particular way. Yeah. And you're saying that if I turned it negative, even though that's physically.
It has no meaning. Yeah. The simulation would continue to change, because it's just a knob that… Well, I don't even think the simulation would change. I don't know if simulation would be run well with negative neutrino maps.
But essentially, just forget about this whole bottom side of the plot. It's not important. Or you should read from this is, like, the site is completely symmetric in the positive-negative direction, so all it's really saying is, like.
This is the size of my Araba or neutrino mass.
The negative thing is just because I'm doing Fisher analysis, but it's not… It's not, uh, really important.
Uh, but okay. But don't you get different statistics if you put in the prior?
As in, if you did a full posterior analysis, you would get some answer, you change the prior, yes, but at this… In this case, we're kind of just looking at the Fisher matrix, which is related to the likelihood under a Gaussian approximation.
Where I've just taken some derivative. Yeah, but if you know it's not supposed to be a calcium approach, I agree with Tom, it's a 4… It's just a misapplication of the Fisher method, because, like, half of your plot doesn't actually exist.
So, it's not gonna… Does it depend on the prior? However, this is a common way of doing it.
We would experiment where you let it… You let it float into the un-physical. Experimentalists do this all. But also, if in the positive neutrino mass regime.
They're, like, half Gaussian is a good approximation, which I think it is, then this is still telling you.
What's going on in the positive neutrina side of things. But in that case, Tom is correct, then your error bars would be a little bit bigger. I don't know about fat analysis is just the most optimistic, uh… This is just a proof of concept. I'm not telling you, like, any numbers, like, this is exactly how you would measure the neutrino mass, but the point is.
Like, regardless of whether this is an exact application or fish analysis, the degeneracy directions are different.
And so there's a lot of information when you combine all the hiring statistics. That's kind of all that needs to be taken away from this slide.
Sorry, I have a question. There's a question from Zoom.
It's just motivational, really.
Sure, yeah, go ahead.
Yeah, sorry I joined late. So, um… Capital M sub nu is the sum of the new addresses.
Yeah. Yeah, that's correct.
Is that correct? But if you can have either sign.
Is that, like, a signed sum of the neutrino of the individual neutrino masses?
Perhaps, but in these simulations, we've only got positive individual neutrino masses, but.
Well, then how did you get negative… how are you getting negative values for the total sum?
But I suppose… This is just the Fisher analysis, correct, that I was saying. Really, if I'd have cut off the bottom half of this claw, there would have been no questions, and everyone would have been happy.
Okay. Alright, fine.
It's really not worth thinking about. It's just, uh, telling you the size of the araba.
Okay, but you're not doing assigned sum of the individual neutrino masses?
That's all, and…
They can measure that correct? Yeah. I can't measure that.
We're just, uh… I have a simulation, I put in a total neutrino mass.
And then I… yeah, I do inference on that value. The sum of the absolute values of the mass, right?
Okay, got it. That's what I wanted to hear, thank you.
Yeah. Okay. Okay, great, let's move on from outside. Yeah.
Okay, uh, but anyway, so what these plots are showing, um… is if I just use the power spectrum, my error neutrino mass saturates at around Kmax of 0.2.
Um, so I don't get any extra information from going to smaller scales, just flattens out, but when I combine with these higher order statistics.
For example, on the black line, when I have power spectrum, halomass function, void size function, this error bond decreases, and it's going down and down and down, even to KMX to 0.5. So this is just motivating the, on small scales.
There's a lot of information about nutrition. Another thing you could look at, instead of just tailors and voids, is other elements of the cosmic web, so you can look at, like, a map of logical structure, classify it as a known filament.
Wolves and voids. So, voids are what we looked at before, nodes is like halos and filaments.
these 1D structures in the map, and what was it 2D structures, you can combine all of them, and also you see that by combining different cosmic web structures, you get good constraints on neutrino mass. So again, this is saying, the cosmic web contains a lot of information about neutrina mats.
But in general, there's many proposals for higher order statistics. I stopped updating this slide in 2021, but there have been a lot more since. And essentially, they're all looking at some different summary of the field.
Uh, for example, like, Bispectrum, which is, like, 3-point correlation function.
Uh, probability density function, some are looking at weak lensing, some are looking at the total matter field, like delta M, some are looking at wretched space, and so the question is.
Um, where is the information coming from? Is it coming from a particular type of summary? Is it coming from a particular, like, view of cosmology, delta M, or lensing? Is one thing particularly good at.
Giving you neutrino mass. And so, uh… To answer that question, let's, uh, over here, we have, uh, Delta M, which I just showed you before.
But what happens if I were to look at Delta C instead?
Then, all the contours line up. And so this is saying that there's no degeneracy breaking anymore, and so the effect of neutrinos on the cold dark matter itself.
It's pretty minimal, and you wouldn't detect neutrino mask very much.
So essentially what this is saying is I need to somehow access delta M rather than delta C.
If I want to measure the neutrino mass with really high precision.
And so what do surveys actually measure? Um, well, they don't measure delta nu, otherwise they'll be able to measure neutrino mass really, really well.
They don't measure Delta C quite, they measure… we measure halos or galaxies, which are biased traces of the cold dark matter. So, in linear theory.
You can think of, like, the Halo Galaxy density field is just… some number, the bias… times the co-docmata density.
But, like, this goes to much higher order, you can have density squared, density cubed terms in this.
Um, and then we also look like measure delta M. We measure something in lensing, this integral of delta M along the line of sight, so you're projecting the 3D to 2D, and you also have these cosmological parameters out the front.
For example, omega M and H0, which if you don't know, will lead to degeneracies with neutrino mass.
So, essentially, it's quite hard to access Delta M, um, and… And so, the question is, like, how can I still measure neutrino mass if, uh, if all I can observe is.
Halo Galaxies, which trace Delta C. Oh, lensing, which, uh, have this projected version of delta M.
And so this, uh, after this little introduction. Is, uh, what motivated me to… Uh, through the reconstruction and the simulation of the universe. So one way we can try and get all the neutrino mass information.
S stand for all of the information from the field, so rather than looking at the summary statistics, actually take the whole field.
And extract cosmological information from it. And one application of that that I've been working on is doing BAO reconstruction using the full field, which I'll talk a bit about later.
And then another thing you can do is jointly analyze multiple redshifts and multiple traces to break degeneracies.
So what I was just showing in the previous slides was just large-scale structure on its own, but if I were to combine large-scale structure with other surveys, like CMB, which tell you about different redshift information, I can use that to get even better constraints on neutrino margin.
And so, okay, yep, sorry. Well, yeah, it's definitely… no worries. I've already said it, but yes, this is what motivated me to simulate and reconstruct the entire universe.
If I just go back one slide. Sure. Ooh, alright. I'm really appropriate everything. I think the back button.
Yeah, here we go. So, here… here's… sorry, the slide before where you showed that everything was degenerate. Yeah.
You get a cold dogs matter. I had been under the impression that, like, the presence of the neutrino, especially in the CMD, is kind of changing.
The power spectrum… because Swipe basically the gravity of the seat of the neutrinos is sufficient to perturb where.
The photon over densities are. which I thought would have changed where… Quite the matter field.
Reservations are showing me that that's not the case.
Where exactly the perturbation? Or the power spectrum of the perturation. It changes, I thought, the growth of some of the.
Yeah, like, so… so the power spectrum will change, even in Delta C. The problem is, like, it's really degenerate with Sigma-8, for example, and so that's what's killing a lot of the information. If I was to fix all my parameters and just look at, like, if I change neutrino mass.
PEC, so it does the power spectrum of C change? It's pretty much every one of these other moments in the same way, which I… Uh, never mind the moments, but uh, yeah, this is showing it does do all… but yeah, once you include all the Lambda CDM parameters and the degeneracies, then it completely kills any information you can get. I thought maybe this is what Matt's saying, it puts a…
A shift in the power spectrum somewhere, and that where that is depends on.
But it's degenerate with the other parameters, and in a way that, uh… Because that feature isn't there.
Yeah, I'm not sure. Um… I don't think it introduces any new feature. I mean, the way I think of it is just kind of shifting omega M.
Uh, a little bit, like, it's transferring… It's a new feature in the… Spectrum.
And the power spectrum of the CDM. Got it.
Clearing his answer, I thought it was degener of Sigma 8 now, so… Looking back, yeah. This does make sense with the more that I have, but I haven't done it. But this is also… that's the overall scaling.
But I think maybe even better answer is just, like, there is still information about neutrino mass here.
It's just… not much. Like, you're not getting extra from all the higher order statistics and information. It's like, the power is… offset in the… in the… shape the power spectrum. That's not included in these things.
I think it should all be included, yeah. And… If it's on the scales of the simulation, then in the initial conditions that we put into the simulation, it would be… it would be initialized with whatever features Neutrino must put on the power spectrum.
Maybe we can talk about this more later, um, but… I mean, there is a direct measurement to try to measure that option.
Okay. Okay, we can chat later about it.
Okay, so now let me, uh, talk about, uh, firstly, field-level inference.
So I showed you this slide earlier, scary negative neutrino masses.
But, uh, this is just with higher order statistics. Um, can I actually find all the information in the field, rather than just, like, using a halo mass function, avoid size function, etc?
Um, and the way to do that is something called field level imprint, so… We can start off in the very early universe, where every pixel, which I've drawn very big, just for illustrative purposes.
is a Gaussian random variable. The initial universe is a bunch of Gaussian random variables.
Um, and then we have forward models, which take the initial universe.
And some cosmological parameters, like sigma A and omega M.
And then give us some final universe, like the galaxies.
Uh, and the idea of field-level inference is to take this backwards. So, given the galaxies that I observe.
Can I infer all my initial pixels of the universe and the cosmological parameters jointly?
And if I can do that, it should give me the most type constraints on the cosmological parameters possible. If I include all the parameters that go into my forward model at inference.
Then it should give me the tightest constraints that I can get.
And so, practically, how we do this is we write down a posterior.
Uh, which is saying, like, uh, given some data, like the galaxy distribution, and some forward model of that galaxy distribution, which depends on these parameters.
Um, minimize the difference between them. And we have some noise here, so it's basically a chi-squared. This is the likelihood, and then you have a prior, which is just saying that the initial universe is a Gaussian random variable.
With variance given by the powerless spectrum. Then you sum this over all pixels in case, and then that gives you some posterior of the initial pixels, the cosmological parameters, given the data that you observe.
But the challenge here is it's a multi-million dimensional parameter space. They're trying to infer every single pixel in your initial universe, which is super expensive.
So, how do we do this? Um, so, best way you can do this is explicit field-level inference, which is basically.
I wrote down the posterior on the previous slide, now let me actually use some methods to infer using that analytical criteria.
Easiest thing you can do is optimization, just find the maximum of the posterior.
Uh, is an animation of that, which I made many, many years ago, because I haven't run optimization for a while.
But, um, if this is your initial universe. And this is the final universe.
Uh, what optimization does is first tries to match the final universe, and then infer the initial universe, and so you can see here, optimization slowly slowly matching the final universe on the bottom right.
And then the initial universe on the bottom left.
But really, in cosmology, we want Araba, so running optimization isn't going to give us that.
And so, what you really want to do is run something like Monte Carlo sampling.
And because this is such a high-dimensional problem. Uh, you want to use gradients to make the cell plane more efficient.
And so the traditional method. People use theirs, Hamiltonian Monte Carlo, so let me just explain how that works a little bit.
So you have some probability distribution, phi of X.
Then it's 1D. Um, Hamilton in Monte Carlo ads in a momentum variable, pi.
Such that the joint distribution is proportional to e to the minus H.
And then, the way you figure out what H is, the Hamiltonian, is you integrate out the momentum by.
Such that it gives you P of X back.
And then, that gives you Hamiltonian, and you can follow Hamilton's equations of motion around the phase space to get samples of the distribution.
And so, you take these steps. Following Hamilton's equation of motion.
And then you get a sample, and then you take a jump, and then you take some more steps, then you take another jump.
You move around, and so one thing about HMC is for it to converge, you've got to take all these jumps in phase space.
So you follow Hamilton's equations of motion for a bit, and you take a jump, and you follow a bit more, take a jump.
And that looks, uh, quite messy. So is there a better way to do this?
And so, one method we've been using here is microprotocol Hamiltonian Monte Carlo.
Where the idea is the same, but instead of using this canonical distribution from statistical mechanics, use the microclonical distribution. So you define the joint.
Probability as proportional to a delta function of H minus E, where e is some energy level.
And so now you can follow these constant energy trajectories in phase space and get samples that way.
Uh, and it turns out this method is much faster, and so if you're doing high-dimensional inference, it's pretty useful, and so showing it in action on a field-level inference.
What uploading here is… yeah, just sitting here doing his quadratic and…
Is the… you're saying the microclinical one? I believe, yeah, so for standard canonical, yes, it's quadratic, it's just B squared over 2M.
in the… in the micro one, I think it's, like, logarithmic in the momentum in some weird way.
Uh, I don't remember the exact equation, but I could show you after. But, um, but essentially, yeah, it's changing the Hamilton, it's changing the way you move around the face space, and it's finding samples more efficiently because of that.
That sounds important. You're changing. for the momentum dependence.
For what the… Subspaces.
Imported in what sense? Like, uh… I picked a different one.
Got a different subspace. Yeah, but I guess, um, essentially, you can make the joint distribution whatever you want it to be, that's totally fine.
But then you've just got to derive the H.
Such that when you marginalize out the momentum. You get the P of XC1. So, essentially, I can choose anything here, it will just give me a different age.
And then I just have to follow those equations of motion. Maybe you could talk afterwards, I'd be curious to understand why.
Okay, sure, yeah. Sure, yeah, okay, let's talk about it later.
Uh, but yeah, in practice, this is showing the… Effective sample size, or this microphone sampler versus the.
HMC sampler, and you can see that as you go to… so this is the efficiency of the algorithm.
As you go to higher and higher dimensions. The efficiency increases and increases.
And so this is, uh, saying that if we go to really, really high dimensions, for example, for cosmological surveys, we're going to need, like, thousands of cube dimensions.
Having a method which scales very nicely with Dimension will be super, super useful.
And so here we were just applying it to delta M, the matter field.
But since then, uh, people have shown that it also works better for bias tracers, so applying it to more realistic galaxy simulations now.
And so here lower is bad, and you can see in this microcronical sampler in green is the lowest, so it's performing the best.
And then recently, we also wrote a paper applying this to the CMB.
their higher is better, and you can see this MC, LMC Micro sampler is faster than the other. So, it's been used in a lot of cases now in cosmology to speed up sampling by quite a bit.
Um, another… The problem with doing explicit build-level inference is just clustering, so when we run an embody simulation, you have a bunch of particles.
And then they collapse into structures like halos, and galaxies form within those halos.
And differentiating through clustering is, uh, very non-trivial to us.
Uh, so a couple days ago, we bought out a pay pub called JFOF, and so this is… It stands for Jack's Friends of Friends, so in cosmology, we call a classifying algorithm Friends of Friends.
Because it's finding particles that are friends of each other, essentially, because they're both each other. I don't know if you use that terminology in particle physics.
But, um, yeah, so we have a JAX implementation of this, which has gradients as well.
Um, okay, but anyway… What are the… Okay, one of the challenges with explicit field-level inference, even with all these nice methods.
is that we don't really know what the likelihood is, so a few slides ago, I showed you… I wrote down the likelihood at a Gaussian form.
But really, like, as you go to smaller scales, like, is it really Gaussian? Is the likelihood I've written down really correct?
And so, to get around this challenge, another thing I've become much more interested in is implicit field-level inference, where you use machine learning.
to learn the samples. So, say you have a bunch of simulations with the initial conditions of the universe, and the final universe, say the galaxies.
You can use a neural network to get. The samples, instead of running something like multicolor sampling.
So one method you can use there is diffusion models, or stochastic interpolence. So let me just explain how that works.
Very quickly. Um, so, say you have some images of Mount Fuji.
You could… you add noise to Mount Fuji, step by step, until you end up with something that looks like pure noise.
And then you could do this for many, many different kinds of images of Mount Fuji.
And then the idea is, you train the neural network to undo this noise. So to go from the noise to Mount Fuji.
And then, if you give the network many, many different types of noise, it should give you many, many samples.
of Mount Fuji. So, just to show this in action.
Here's a picture of me and my collaborator, Carol, at Mount Fuji, but it was very, very foggy, and we couldn't see Mount Fuji, and we were very sad.
But with a diffusion models.
Uh, but you don't have to just train this algorithm on Mount Fuji. You could actually train it on simulations of the universe. You look kind of different, too.
Yeah, well, we look even more different. Because the universe is so exciting. Yeah, so essentially what you're doing is, uh, mapping between the noisy.
Initial conditions of the universe to. the final universe.
And so, what we actually did, uh, in this paper for 2024 is, like, asking how many simulations do I need in order to train a diffusion model, and how does it compare to something like Hamiltonian Monte Carlo.
And so, what this plot is showing is this R of K is showing you the quality of the inference, so higher is better.
Uh, different colors, just different scales of the simulation, but the main point is, with a stochastic insert, the solid line.
Uh, you get this kind of convergence as a function of number of simulations, and then the dashed line is HMC, and you can see it takes around 10 to the full.
Also, simulations in order for the two to converge to one another. So that gives you an idea of.
If I really wanted to do it, place it for level inference with a diffusion model and get samples.
That's how many sims I would need. For each sim, are you training it on little subsets of it, or it's only 10 to the floor… Uh, instances that you tested for. So this is a, like, so we're using relatively small boxes here, so we're running on the full simulation each time, so it's 10 to the full actual simulations.
We're training with the whole simulation. Oh, that's not a lot of… Instances, I guess, for training a diffusion model with how many dimensions is it? Uh, 64 cubed is the highest we did.
Yes, it definitely, yeah, I think scaling it up to 256 cubed or higher, that's a challenge. Also, in terms of memory.
But, uh, but yeah, for… Like, for the patients we do, we usually have millions or.
Tens or hundreds of millions of instances. How many dimensions is that much? Even lower dimensions, just 100 dimensions.
Yeah, interesting. And how do you quantify, like, a number you need?
So here we were, I don't think we've made a plot that literally like this, but we… just to get sort of a good, um… fidelity to the training set.
Okay, um, what we would consider, like, an acceptable fidelity.
I see. Interesting. Okay, it'll be interesting to see that exactly. When they trade diffusion models on images to do, like, the stuff you just showed.
We hear training on way more than 10,000 images, right? Yeah.
Yeah, maybe it could be this task is maybe a bit simpler, is mapping from, like, some linear density field to a… Uh, just, like, non-linear density field, it's not as complicated.
Of a shift to learn. Could also be this… so here we're using stochastic interpolence, rather than.
The diffusion models, perhaps they're more… Uh, simulation efficient.
But I didn't think so either. Yeah, okay, interesting. It'll be good to chat about, you know, applications later.
And so this is what we're finding in the cosmological applications we've been looking at.
But another thing that I'm excited about now is just using a CNN as a point estimator. So instead of getting samples of a distribution.
what if I just take my galaxies, and I just map them to the initial conditions with CNN?
Um, and specifically, we've been using this to improve BAO reconstruction.
So just to give you a very quick one-slide overview of PI reconstruction, this is the results from DESI dates release 2.
Uh, and essentially what Desi's measuring is these distances relative to lambda CDN.
Uh, in this blood versus, uh, time or distance away from us, for a bunch of different galaxy samples in different colors.
And it's measuring a two-point correlation function and this peak in the two-point correlation function in the BAO peak.
And just for measuring this peak, you've learned a lot about cosmology.
And so, the way this is done is using a method called BAO reconstruction.
So, in the early universe. Think of this, this is a VAOP, looking, like, top-down, and here is actual, like, side view of the peak.
This is the early universe, when everything was nice and clean and uniform, then non-linear evolution takes place.
The peak starts getting spread out, and everything starts getting smeared out, because of not only the evolution.
And then, traditionally, you do something like perturbation theory to undo.
this smearing of the BAO. And try and reconstruct the initial universe.
But using perturbation theory doesn't quite reconstruct things perfectly. You can see, for example, this black line after reconstruction doesn't quite hit the dashed line.
And so there's room for improvement. In terms of reconstructing the video.
So, what we've been doing… is using a CNN to improve the reconstruction, so you start off with some simulated galaxies. This is now much higher dimensional problem than previously, so here we have.
256 cubed and a 1G of plastic box. So, uh, yeah, uh, and then you do the… traditional BAO reconstruction on the galaxy field, so this gives you the correct large-scale reconstruction behavior.
So you've reconstructed the large scales using perturbation theory there.
And then the idea is, you feed both of these into a CNN.
To predict the true, uh, initial. Or a linear field. And so the reason for giving the CNN both the BAO reconstructed field and the galaxy field is because.
If you give the CNN the correct life-scale reconstruction, it doesn't have to work hard to figure out the large scales, it can just focus on the small scales.
So it's much easier to trade. And it also means you can use a subgrid approach instead of.
training of the whole book. So, even if I have, like, uh, say I have a 1Gap box, I just train on that.
I could, uh, apply my CNN just on this little sub boxes, so say 200 megabyc is what we used here.
And then I can just tile my entire volume with these sub boxes.
And even if I trained on some volume, I could just scale it up to any volume with no loss of performance.
And so let's have a look at this in action.
So this is now showing the cross-correlation coefficient, so again, the quality of the reconstruction, where 1 is what you want to be getting.
As a function of scale. Um, we have 3 different line styles for different angles to the line of sight. This is all our spreadsheet space distortions, change thing in the.
The direction radio from us. And so you have different performance depending on the line style.
But yeah, in gray, you have the standard reconstruction, in orange, you have the CNN corrected reconstruction.
And it blew you just have if there was no register space distortion, so that's the best we could possibly do, and you can see CNN is giving us better reconstructions than the traditional method.
And now, this is when I trained on 1 gigaparsec and tested on 1 gigap, but… what happens if I test it on a 3GB of plastic box?
Um, so here, the cellulite is 1 gigaparsig, the dotted line is 3 gigaparsec, you can see solid and dotted overlap with each other, so it works perfectly. There's no loss in performance as they scale this method up to higher volumes.
So this is great. I can scale this up to survey volumes.
And so, what I've been looking at recently is actually feeding us through the whole DESI pipeline, measuring the BAO scale, so we're looking at the parallel and perpendicular scale of the VAO.
getting constraints on these parameters. Another way you could summarize these, uh, the BAO scale is with these parameters called QAP and QISO.
Uh, not important what they are, but the take-home message is.
Um, with solid reconstruction. For example, this QISER parameter, I get a 0.01 error bar with explicit field level inference, so writing down my likelihood, doing the HMC.
I get a 4% improvement, but with the CNN, I get a 16% improvement.
And then, 16% both in this parameter and this parameter, so it's kind of like 16 squared.
And so, that's pretty exciting, and we're now, like, scaling this up to actually apply it to all the different Galaxy.
Uh, measurements from Desi. And what does this mean for cosmology?
So, like I showed you earlier, if I could measure.
this pink contour more tightly. It should give me a tighter bar on the neutrino mass.
Um, but looking further forward. Uh, another reason why we want to be.
Um, like, yeah, another thing you can do with BAO, for example, is WNOW and WA.
If you just looked at Desi on its own, uh, you get these constraints, which is really not that exciting, you know, measuring.
Uh, no, you're not detecting anything, but it's when you combine with the CMB, and you get these contours which differ from the.
Now, you know, minus 1 and WA of 0.
So really, the excitement is coming from the CMB across large-scale structure, and just like I said over here as well, really, even though you're measuring the VA only better.
As the CMB cross-correlation, which is giving you the information on your treatment list. So really.
What we should be doing in cosmology is combining the large-scale structure and the CMB data, like, fully at the field level.
Uh, rather than just doing a BAO and combining it with CMB.
And so this is another project that I'm working on, where we're making… So before you go to the other. Yeah, yeah. So, here you did field level, you go back to the nice.
But, you know, for the… Yeah, so… Yeah, so… I can ask my question again about the robustness of the simulation, so this using… what simulations using? Sure, yeah, so now, yeah, so now this is a good question, so…
We're using, uh, like, uh, in this particular analysis, yeah, we're using end-body simulations, put on HOD galaxies.
Which ones? So, Quahote is what we use, and also we have a FlaskPM version as well.
But uh… but yeah, but I think the main problem, really, is the Galaxy population, the HRD. Like, how do you populate your galaxies? Does that match the real… so it's Halo occupation distribution. Okay. So the idea is to… do clustering on your math particles, you get halos, and then there's some probability that galaxies with a certain property live inside those halos. That's how HOVs work. And then there's extensions to HRDs, where you add extra properties to make it more complex.
Um, so… but okay, those are technical details. But, so, it's true, like, uh, when you do this kind of supervised learning, you're training on some sims, and there could be robustness issues.
But, um… so, in general, food-level inference, that's a risk of it, but one reason I'm very excited about BAO reconstruction in particular for field-level inference is because.
It's been shown to be really robust, so even when you do traditional behavior reconstruction.
you assume a cosmology, and you assume a covariance matrix, for example, based on simulations. And people have shown, even if you get it slightly wrong.
you still recover the correct BAO scale. And so we've been doing the same tests that are done in traditional VA reconstruction.
So, like, what if I simulated different kinds of galaxies? What if, like, the bias I assume is wrong? What if I assume the wrong cosmology, and we still get robust, uh.
constraints on the VAO. There could be a slight, like, increase in the air above, but yeah, one of the really nice things about applying this to BAO is it's a very robust.
Submary. And that's why, I guess, Desi is really excited about using the BAO specifically.
So, but you're using this CNN supervised on a particular choice of HOD simulation, to sharpen the VAO, yeah.
And that's responsible for the 16% gain that you're showing. Yeah.
Have you done… I guess you've done tests where you trained it on one HRD, and then you applied it on another HRD, and you still get 16%? Uh, so… I haven't phoned through to the 16% level, but what we have in the paper is, um.
Just looking at this flow. Like, so the 16% is kind of related to how far away the orange is from the green. The higher the orange is, the more the difference between the two would be.
What we found was, um, the orange goes down a little bit, but not so much so that it agrees with the gray.
Even in a really extreme, uh, like, different business model. Yeah.
And what… how do you understand that? Like, why… why is it… Why is the sharpening… like, trained on a different galaxy. Why is it so robust?
I guess because we're still kind of on scales where it doesn't have a huge effect.
Um, and um… and then I guess the way you do BA reconstruction.
You, uh… when you fit the BAO, so once I've reconstructed my field.
I measure the two-point relation function in that field.
And then I… then I fit, uh, something to that.
And then when I do this fit, I marginalize over a bunch of nuisance parameters.
Which, I think that's where a lot of the mispecification is coming into play. Not necessarily on the peak itself.
But just in other aspects of the power spectrum which get marginalized out.
So yeah, you're saying that the specification gets, um… Smeared out, or accounted for later, like, downstream?
Yeah, as in, through the pipeline, at the end of the day, I'm just trying to fit the peak itself. And the peak itself is somewhat robust due to these specifications. If I get the wrong cosmology, if I get the wrong.
Biasmodo. Yeah, and that's why… I mean, sure, it's not perfect, but it seems that.
Yeah, uh, you can still do a pretty robust analysis. Do you just want your location, or also the height?
Every location, uh, shape, the full shape is what you're interested in.
Well, yeah, 5, yeah. That's… that's what we're finding.
Okay, but it's a purely, like, vertical result. It's purely empirical, and even in Desi's main analysis with traditional methods… The traditional method was also empirical as well. Yeah.
Yeah, it's very… it's… I mean, I agree with your question, but uh… yeah, seems to be quite a robust statistic. That's why it's got a little bit of attention.
Sure. Okay. Okay, so… I guess I only got a little bit of time left, so let me really rush through this. So, another thing I've been working on is making simulations.
which contain… which are really, really big, so they contain CMV all the way down to the large-scale structure.
And also, they model these simulations of all Half Dome.
Um, yeah, so they model the ways in which the CMB interacts with the large-scale structure.
So, you have CMB plus the 3 universe, goes into your detector.
But it interacts with the large construction in various ways. For example, we have CMV lensing, where.
The light bath is bent by the matter on its way to us, and this bending tells us about the matter.
That is passed through. Another very exciting effect is the SC effect, kinetic SC and the thermal SC effects in Javowich effects, and this is the interaction of the photons.
with electrons, it's both inside galaxies and outside galaxies, and this is really interesting because it tells us about what's going on inside the gas around galaxies. If you can see the effect that this has on the CMV photons when you observe them.
They can tell you about what's going on inside gout.
Uh, and then there's also just the fact that galaxies emit infrared radiation, for example. It's a range of radiation.
And this is a contaminant of the CMB, essentially, which you want to separate out when you're measuring the CMB.
And so, uh, with upcoming surveys like the silence observatory.
Uh, we're going to be able to measure these different effects, but we need simulations which actually model things.
on the small scales, uh, so that we can, you know, test our pipelines and do simulation-based inference.
Um, and so… Uh, just because I'm running out of time, essentially none of the simulations that are available quite tick all these boxes.
But in half dome, we have these very large boxes with lots of particles.
Hitting, kind of, the mass limit needed for DESI, uh, LRG galaxies, and we're simulating all the different observables.
Currently, we have 11 realizations of these at a fixed cosmology, and one with.
primordial node Gaussian, but I'm in the process of running more with different neutrino mass and different values of W.
Um, and we have this great team of people who are making this possible with experts in the CMV world, in the weak cleansing world, and large-scale structure world.
Um, and so really, the whole point of these simulations is to make it, like, multi-surveyed.
Multipro, because that's the direction, really, cosmology is going in, I think, in terms of combining.
all these different data sets. Just to understand… the advantage here, so, like, the CMV that you're gonna get.
is consistent with the… The universe in which you're living in, so it's consistent with where the galaxies are. Yeah, as in, yeah.
The main thing that you just had to do is when you sampled.
The initial conditions you're sampling on a larger scale than you strictly needed to for the box?
It's not really about, uh, that, really, it's more these effects, like the KSD and the TSC and the weak lensing.
You have to, like, have some matter. Which the light of the CMV is going through, and so the point is, you're just tracing that light through the matter, and that matter is the same matter which made the galaxies, and so everything is…
correlated, um, in that sense. But yeah, in order to accurately simulate CMV lensing, you also need a very large box, because.
You've got light passing through matter, which is really hard. Right, and we need the matter all the way out to the COB. Yeah, well, not all the way to the CMV, because that would be crazy, but you go as far as, like.
As far as you need to encapsulate all the non-linear effects.
And then you could put everything else just linearly.
So I assimilate to as high a redshift as I need, such that things, uh, such that all the nonlinear clustering is.
Taken into account, and then everything else you can just use linear theory to fill it.
Just back to the envelope, but you do need to go out.
What's he? Uh, so it depends on the probe. Uh, for… There's for, like, SA, uh, around 4 or so is good, so you have 4. The CMV lensing, some people say it's good to go to C of 9, but…
Yeah, it depends how small a scale you really want to go to. But between 4 and 9, essentially.
Okay, so here's a… okay, an image of the CMV from our simulations, the lens CMV. I'm zooming in on a 5 degree patch here.
Um, in between the CMB and us, it's all this matter, which I mentioned earlier, so this is the MATA integrated along the line of sight, the Kappa map.
This is responsible for weak lensing. And then, when the light from the CMV passes through the matter, when you have the thermal cellovich effect, the thermal SE effect.
It lights up the gas around the galaxies. That's all we're seeing now is the gas around the galaxies lighting up in this thermal SC map.
And then the kinetic SC. is, um… it has a direction to it, because the Connecticut is tracing the velocity of electrons, or the velocity of galaxies, and so you have a positive direction, negative direction.
Uh, that's what the blue and the red represented. So, these animations hopefully show you that everything is correlated, so the same matter, which is responsible for lensing, is the matter which is responsible for the SC.
And so now you can start doing these combined probe analysis with these kind of simulations.
Uh, and then another somewhat correlated progress is that cosmic infrared background. So this is super noisy. It actually is correlated, but it's very hard to see, because it's really noisy. And one of the big problems in CMD analysis is actually separating this out.
From the observations, such that you can actually see the TSC.
with KC, and this is something I'm happy to chat about.
Um, but just, uh, I guess in my last few minutes, another thing I wanted to tell you about is numerical foregrounds.
Which David's been asking about a lot. So, uh, I showed you, like, the CIB, that's a physical foreground, right? It's blocking the observation of the CMV. There's also numerical foregrounds.
Uh, like, the size of the books that you use to run the simulation, I guess I just answered that question to Matt.
Having a book, so how high regards you need to go to. Robustness, right? If you're using different simulations, what do you do? Also, how many simulations do you need? I'm going to skip over block size, because that's maybe not so interesting to this crowd.
But let's go to robustness. Okay, so let's do a field-level comparison of in-body simulations.
So I have all these different simulations, Gadget, Abacus, QB3M, Enzo, Gizmo, etc.
They all kind of look the same. So these are 2D cutouts, and then here is a zoom-in.
On the 2D cutouts. Uh, they all kind of look the same by eye, but if I was to take a ratio of each one… How do you make them look the same? You're starting from exactly the same initial… exactly the same initial condition, exactly the same cosmology. Could be the same gravity, yeah.
Yeah, but not quite. So they all… when you take the ratio of each one with respect to gadget.
Uh, you get these, you know, non-one ratios. And in particular, one that stands out, just to explain why, I mean, this is understandable.
Ramses versus Gadget, for example. So, the way Gadget works is it has a, like, a cubic grid to solve the forces.
was Ramsey's uses something called adaptive mesh refinement. So, it adaptively refines the mesh in high-density regions, and it kind of doesn't refine the mesh in low-density regions.
So… so Ramsey's gonna have much more highly resolved, uh, high-density regions, the Gadget, a much smoother low-density region of the gadget. If you really, uh, squint.
You can see that the void here, this purple blob, is a little bit smoother in Ramses.
But anyway, the big question is, if I was to do field-level inference on these simulations, like, is one robust with respect to the other?
So if I train on Gadget, and then I test all the other simulations.
Uh, things robust. And so what we find? Here, it's uploading the info at value minus the true value.
Predicting omega n and sigma A. And so, if things are robust, unbiased, you want a horizontal line, and you can see that in gray, gadget.
When I test… train it on Gadget, test on Gadget, it's flat, so that's great. Gadget agrees with Gadget. Abacus, uh, QB3, etc, they're all doing a pretty good job.
But with Ramses is super biased. This purple, uh, purple icon. Interestingly, uh, in blue up here, we also tested on TNG, which is a hydrodynamic simulation. So this has got, like, all sorts of complicated gas going on, and it's actually less biased than Ramses.
So this is telling us that these kind of adaptive measurements that Ramses is doing is.
very much confusing the CNN that have trained on gadgets. Why is it last? Isn't the blue higher than the purple? Uh, the height is irrelevant, this just shifted for a fact, but essentially you're just comparing the.
The Arab eyes to the horizontal line of the same color.
Oh, so it's just, yeah, just the fact the purple error bars are really high above the purple horizontal line means it's bias.
Oh, okay, yeah, yeah, I understand. But you'd like it to be scattered around the horizontal line. Exactly. There's, like, one. Yeah, purple is quite… Quite, boys, yeah, exactly.
So what's going on here? Um, so here's some projects I did with some students, uh, trying to interpret what CNNs are learning when they're being trained on.
density fields in cosmology, so here's the image of the density field.
And we're just training CNN on this to predict omega F.
And you can do these things with integrated gradients, where you measure the response of your network output with respect to the input.
So you're basically taking a derivative. Uh, through your neural network, and seeing, like, if I change the value in this pixel, how much does that impact over again?
If I change the value in another pixel, how much does that impact them again?
I think… I think they don't like robustness outside.
So, anyways, you get a map that looks like this, integrated gradient.
map, and so that means neural networks paying a lot of attention to that region, like the gradient is high.
And white means it's not paying much attention to, so you can see black over here, it matches up with this high-density region, black over here matches up with the slow density region. In the paper, we actually made a histogram of this gradient as a function of density.
And you can see that the neural network's really paying attention to the most high and least.
high, or the lowest density regions. And so it makes sense that if the network is paying, like, most attention to these really high-density and low-density regions, if you have a different way of modeling the high-density and low-density regions, it's going to be.
robustness issues. And so, really, the easy solution to this explanation, is just.
smooth your fields before you train the CNN, uh, and so what we showed was that if you just apply a simple, like, Gaussian smoothing on your maps.
Uh, everything becomes in distribution as far as omega and sigma are concerned. So, here I'm showing the constraint on omega m, but different smoothing scales. At the start, there's no smoothing, so you see.
these different versions of Gadget and Ramses disagree with each other, but then as they smooth and smooth and smooth.
They start to agree and have the… agree with the true value. And the error has increased.
Because that's moved, I've killed some information, but things are now robust. And so, essentially, what we were simulating here was.
Uh, grid size of 256 cubed, and we're finding that you need to smooth by around 6 times the grid resolution in order to make things robust.
Um, so this isn't crazy, like, if different simulations use different, like, algorithms to model the smallest of scales.
You can't trust them all down to the smallest, smallest scale that they're probing. You've got to smooth them.
Uh, at some point. And so that all we're doing here is showing, like, what, how much you have to smooth fields for them to agree.
Instead of, uh… so what I've done now is I've just done my inference, I've shown you it's biased, I've then gone and smoothed everything, and I've shown you how I can make things unbiased.
But another cool thing we did in this paper was assess the robustness at the data vector level.
Well, hold on, I mean, but yeah, you can smooth it to make it unbiased, but then have you wiped out all the benefits of training at field level?
So, I don't think so, no, not in the… Um, there's still a lot of information based on this error, but we haven't done that exact comparison, comparing it to the power spectrum, but I'm pretty sure there's still a lot of information from the field level.
Um, and the point is, you could always, like, simulate more and more and more and more particles, or higher and higher, higher resolution simulation.
Um, you just have to cut out part of that resolution from the simulation.
So, I guess the way I think of field-level biow spectrum is.
Uh, some Kmax, no matter what my simulation is, field level's gonna give me more than the power spectrum.
At that Kmax, regardless of the simulation. But in order to reach that.
That, um, scale robustly with the simulation. I have to simulate to even smaller scales, and then smooth such that I reach.
that scale that I'm interested in using to level influence.
I guess, yeah, point is, I can still simulate accurately down to the smallest of scales, it's just I've got to.
go a little bit beyond the scale that I'm interested in.
In order to be robust. Okay, okay.
Um, okay, um… Assessing robustness at the field level, or at the data vector level, there's a method called PQMass.
Where… which you can just take two sets of samples.
And see if they're in distribution with each other.
So here, I'm just showing this in 2D, so imagine I have a 2D data back to instead of my whole field, which is really, really high dimensions.
I could split my parameter space into many little boxes.
And then I have some samples from Gadget say, which are red, and some in blue, and I have some samples from another simulation, which are in red.
I can look in each of these patches, count how many samples there are in each patch.
And then I can define a chi-square test. To say whether these two samples.
come from the same distribution. And it's really, really quite simple.
Multinomial distribution chi-squared test. And so, at the end of the day, what you end up with is these kind of plots, where you have, like, some predicted.
chi-squared, which is the histogram, and then the expected chi-squared, which is the solid line. And so gadget.
agreed for gadget. You can see that the histogram lines up with the… With the light, but then all the other simulations are shifted a little bit, because they're out of distribution, and Ramsey's more so than…
than the ones that are left. Um, and then when you smooth, you see everything becomes in distribution. So this is a very nice method of.
Without doing any inference, just checking if your simulations are in distribution with one another, even at really, really high dimensions.
I think we would have, uh, done this a different way, which we would have trained a classifier.
On visual samples. Mm-hmm. The distributions are the same.
Yeah, okay, but that's also quite expensive, the trade classifier. Here, there's no training or anything.
Just, uh, you do this really simple chi-squared test.
It sounds like you're doing some kind of formula's version of density estimation with your… Yeah, but because it's cool, man, it's really cheap, and really still effective.
But, uh, but yeah, essentially, that's kind of what you're doing by counting the number of things in each.
Look, is this… is each bar… is it each… Entry in the histogram a simulation? Yeah. Okay, perfect.
We have a bunch of gadget simulations and a bunch of.
Yeah. Of the same initial… of different initial conditions.
Different initial conditions, but yes, if… Mm-hmm. Okay.
Okay, so this is a cool method for doing that. Um, one other thing I'm interested in nowadays is transfer learning. Um, so… Say I want to do inference on a very expensive M-body simulation, I want to test some beyond CM cosmology, for example, F of R, modified gravity, or neutrino mass, or something which is very expensive to run.
can I train my network on a fast simulation, like FastPM?
And then, on Lambda CDM simulations. take the weights from my neural network and transfer it.
to this more expensive simulation to save. simulation budget on the expensive simulations.
Um, if you really want to go down the foundation model route, you could say that we're training a foundation model with cheat simulations.
But yeah, so recently, I've been playing around with this. One example is just going from low fidelity to high-fidelity simulations, so from fast.
It's a more expensive simulations. And what this point is showing is the negative load test probability, so this is, like, the training loss lower is better.
Uh, as a function of the number of simulations used.
If I just train in high-fidelity simulations in red, you can see it goes down like this.
But if I use weight initialization transfer lighting, so I take the weights that I first pre-trained on my cheat simulations, transfer it to my expensive simulations, I get the blue line.
And it's much cheaper to trade. And then we also proposed another way of doing transfer learning, which is even cheaper.
Um, so this is very promising and going from.
Uh, low to high fidelity. And then, in terms of going beyond the standard model, so if I just trained on Lambda CDM, and then I want to do inference on some cosmology.
Which country… which contains neutrino mass and modified gravity F of R.
If I went straight to the complicated cosmology, I get this black line, so the MSC is decreasing as a function of simulations, but if I first pre-trained on 2,000 lambda CDM simulations, I get the blue line.
I pre-trained on 22,000 Lambda CDM simulations, I get the red line, and so you can see this pre-training approaches.
It's helping quite a bit. Uh, so that's pretty exciting, I think, when we move forward to testing lots of different cosmological models with upcoming data.
Uh, okay, so we're almost wrap up. Uh, I guess, yeah, so I showed you earlier.
If you just have electrical structure on its own, you get these constraints of W0, WA, which aren't that exciting.
Then when you combine with the CMB, you get even more exciting constraints, um… And similarly, for neutrino mass, a lot of the information is coming from combining molecular structure with CMD.
So, hopefully with CMV, cross-LSS, plus field level inference, plus noble statistical techniques, we can get even tighter constraints on these parameters and actually detect.
Uh, neutral mass and evolving dark energy. I would usually end off now, which maybe I should because of my late, but what about plus AI agents? So just to… I was speaking to some people about this, uh, at lunch, but I'll just leave this there at the end. So, this is this multi-agent system that I've been working on.
with various folks at CCA, where you can give us some data, a description of the data, and it can do various things, like make ideas, write papers, etc. It's open source.
If you're interested in playing around with that, feel free, feel free to ask me questions about it.
Uh, thank you very much.
For many people have been used. How many, what, so your newest paper have you used your brother is mothered because of… what's some interesting experiments we didn't do is, like, for example, with one of the studio projects I mentioned, yeah.
The day before we put it on archive, we asked the AI to try and make the paper.
Without, like, you know, just see how it would compare to the student.
And so, if you gave the AI a really, really, like, long prompt.
It would actually do a pretty good job, but they're kind of doing a lot of the work for us. Like, we kind of gave them the results.
If you gave it very, very little prompt. Uh, it actually did some really, really funny things. One funny thing which particle physicists might appreciate. So, uh, we were training at CNN.
And so you have, like, a diagram. You could draw a diagram of a CNN architecture, so, like, a block, like a square block, and a smaller one, a smaller one, smaller one, then you have, like, an MLP layer at the end.
And so, basically, we gave the AI the figures already.
Um, we asked it to write a paper. And so, given this, like, image of a CNN, the caption that it wrote was.
A scintillation detector, and surviving a scintillation verbi. I thought it was a particle physics.
So yeah, it, uh… you gotta give it some help, I guess.
I have a dumb question, like, sure, that… I don't know, maybe if you know the answer or not.
So, all this is… And body simulations are Newtonian, right?
When we should start to worry about basic effects and all these things? That's a good question, actually, and I think it becomes.
relevant on, um, on the really large scale, so when you're running these.
CMB course simulations that I was mentioning, when you go to really large volumes.
Then you've actually got to start taking that into account. One very cheap way of doing that, which.
It's what we've been doing, is using a different gauge in your initial conditions, which approximately takes that into account.
Um, yeah, that's as far as I'm aware. Okay. Whether… I don't think we'll be able to detect anything like that soon, but there's a lot of papers about this I could refer you to. Yeah, because I know it… Euclid is worried about… not worried about this, but yeah, but like, it's kind of like, yeah, like, maybe they're gonna have so much data that we can still start to see the ability to see fit.
Yeah, I think for large volumes, it's definitely important, yeah.
Any more questions? Okay, so… yeah, let's take Abram again, or anything cancel.