Heidelberg Institute for Theoretical Studies
Set-LLM, developed by researchers at HITS and KIT, introduces architectural modifications to Large Language Models (LLMs), enabling them to process mixed set-text inputs without sensitivity to element order. The framework achieves robust, permutation-invariant performance, consistently matching or outperforming traditional baselines and computationally expensive methods while maintaining the efficiency of a single LLM inference.
We present new deep imaging and high-resolution spectroscopy of the extreme-abundance-discrepancy planetary nebula Ou 5, together with photoionization modelling aimed at probing its unusual thermal and chemical structure. The nebula exhibits a nested bipolar morphology, including inner and outer shells, faint outer lobes, and polar knots. Remarkably, all these components share a dynamical age of order 10,000 yr. Thermal broadening of the H alpha line relative to heavier ions implies a hydrogen temperature 3000 K to 6000 K, in contrast to the ~ 10,000 K derived from collisionally excited line diagnostics. This provides independent support for the presence of at least two distinct temperature/metallicity phases, as previously proposed to explain extreme abundance discrepancies. Photoionization models with sinusoidally varying metallicity successfully reproduce the observed nebular spectrum and morphology. A mixture of fluctuations with both extreme and moderate metallicity contrasts is required to simultaneously fit the O II and the [O III] observations. The nebular He II emission demands a hotter and more luminous central star than previously inferred, consistent with a ~ 0.58 solar mass post-AGB progenitor evolving toward a CO white dwarf. Ou 5 thus reinforces the link between close-binary nuclei and extreme abundance discrepancies, and provides a valuable benchmark for understanding how common-envelope ejections give rise to the thermal and abundance inhomogeneities observed in planetary nebulae.
Deep generative models have recently been proposed for sampling protein conformations from the Boltzmann distribution, as an alternative to often prohibitively expensive Molecular Dynamics simulations. However, current state-of-the-art approaches rely on fine-tuning pre-trained folding models and evolutionary sequence information, limiting their applicability and efficiency, and introducing potential biases. In this work, we propose a flow matching model for sampling protein conformations based solely on backbone geometry - BBFlow. We introduce a geometric encoding of the backbone equilibrium structure as input and propose to condition not only the flow but also the prior distribution on the respective equilibrium structure, eliminating the need for evolutionary information. The resulting model is orders of magnitudes faster than current state-of-the-art approaches at comparable accuracy, is transferable to multi-chain proteins, and can be trained from scratch in a few GPU days. In our experiments, we demonstrate that the proposed model achieves competitive performance with reduced inference time, across not only an established benchmark of naturally occurring proteins but also de novo proteins, for which evolutionary information is scarce or absent. BBFlow is available at this https URL.
We present the CIENS dataset, which contains ensemble weather forecasts from the operational convection-permitting numerical weather prediction model of the German Weather Service. It comprises forecasts for 55 meteorological variables mapped to the locations of synoptic stations, as well as additional spatially aggregated forecasts from surrounding grid points, available for a subset of these variables. Forecasts are available at hourly lead times from 0 to 21 hours for two daily model runs initialized at 00 and 12 UTC, covering the period from December 2010 to June 2023. Additionally, the dataset provides station observations for six key variables at 170 locations across Germany: pressure, temperature, hourly precipitation accumulation, wind speed, wind direction, and wind gusts. Since the forecast are mapped to the observed locations, the data is delivered in a convenient format for analysis. The CIENS dataset complements the growing collection of benchmark datasets for weather and climate modeling. A key distinguishing feature is its long temporal extent, which encompasses multiple updates to the underlying numerical weather prediction model and thus supports investigations into how forecasting methods can account for such changes. In addition to detailing the design and contents of the CIENS dataset, we outline potential applications in ensemble post-processing, forecast verification, and related research areas. A use case focused on ensemble post-processing illustrates the benefits of incorporating the rich set of available model predictors into machine learning-based forecasting models.
Dynamic graphs exhibit complex temporal dynamics due to the interplay between evolving node features and changing network structures. Recently, Graph Neural Controlled Differential Equations (Graph Neural CDEs) successfully adapted Neural CDEs from paths on Euclidean domains to paths on graph domains. Building on this foundation, we introduce Permutation Equivariant Neural Graph CDEs, which project Graph Neural CDEs onto permutation equivariant function spaces. This significantly reduces the model's parameter count without compromising representational power, resulting in more efficient training and improved generalisation. We empirically demonstrate the advantages of our approach through experiments on simulated dynamical systems and real-world tasks, showing improved performance in both interpolation and extrapolation scenarios.
The growing importance of intraday electricity trading in Europe calls for improved price forecasting and tailored decision-support tools. In this paper, we propose a novel generative neural network model to generate probabilistic path forecasts for intraday electricity prices and use them to construct effective trading strategies for Germany's continuous-time intraday market. Our method demonstrates competitive performance in terms of statistical evaluation metrics compared to two state-of-the-art statistical benchmark approaches. To further assess its economic value, we consider a realistic fixed-volume trading scenario and propose various strategies for placing market sell orders based on the path forecasts. Among the different trading strategies, the price paths generated by our generative model lead to higher profit gains than the benchmark methods. Our findings highlight the potential of generative machine learning tools in electricity price forecasting and underscore the importance of economic evaluation.
Recently, there has been a growing interest in designing text generation systems from a discourse coherence perspective, e.g., modeling the interdependence between sentences. Still, recent BERT-based evaluation metrics are weak in recognizing coherence, and thus are not reliable in a way to spot the discourse-level improvements of those text generation systems. In this work, we introduce DiscoScore, a parametrized discourse metric, which uses BERT to model discourse coherence from different perspectives, driven by Centering theory. Our experiments encompass 16 non-discourse and discourse metrics, including DiscoScore and popular coherence models, evaluated on summarization and document-level machine translation (MT). We find that (i) the majority of BERT-based metrics correlate much worse with human rated coherence than early discourse metrics, invented a decade ago; (ii) the recent state-of-the-art BARTScore is weak when operated at system level -- which is particularly problematic as systems are typically compared in this manner. DiscoScore, in contrast, achieves strong system-level correlation with human ratings, not only in coherence but also in factual consistency and other aspects, and surpasses BARTScore by over 10 correlation points on average. Further, aiming to understand DiscoScore, we provide justifications to the importance of discourse coherence for evaluation metrics, and explain the superiority of one variant over another. Our code is available at \url{https://github.com/AIPHES/DiscoScore}.
36
Eddington ratio is a paramount parameter governing the accretion history and life cycles of Active Galactic Nuclei (AGNs). This short review presents a multi-faceted view of the importance of the Eddington ratio spanning varied AGN studies. We find that the Eddington ratio is crucial for standardizing the Radius-Luminosity (R-L) relation - a necessary step for employing quasars (QSOs) as standardizable cosmological probes to help clarify the standing of the Hubble tension. In this data-driven era, we consolidated disparate aspects by developing novel relations borne out of large datasets, such as the robust, nearly universal anti-correlation between fractional variability and Eddington ratio derived from Zwicky Transient Facility (ZTF) data, which is vital for interpreting forthcoming high-cadence surveys like Rubin Observatory's LSST. Addressing the conundrum where JWST results suggest an overabundance of massive high-redshift black holes, we demonstrate that local AGNs offer clarification: Changing-Look AGNs (CLAGNs), driven by rapid Eddington ratio shifts, cluster in the low-accretion regime, a rate independently confirmed by our integral field spectroscopy and photoionization modeling of a well-known Seyfert 2 galaxy, rich in high-ionization, forbidden, coronal lines. Conversely, for the high-redshift, high-luminosity population where traditional reverberation mapping (RM) is highly impractical, photometric reverberation mapping (PRM) offers a rapid alternative to constrain accretion disk sizes, enabling efficient estimates of black hole masses and Eddington ratios. Finally, we developed tailored semi-empirical spectral energy distributions (SEDs) for extremely high-accretion quasars, successfully validating their characteristic extreme physical conditions.
We provide a brief, and inevitably incomplete overview of the use of Machine Learning (ML) and other AI methods in astronomy, astrophysics, and cosmology. Astronomy entered the big data era with the first digital sky surveys in the early 1990s and the resulting Terascale data sets, which required automating of many data processing and analysis tasks, for example the star-galaxy separation, with billions of feature vectors in hundreds of dimensions. The exponential data growth continued, with the rise of synoptic sky surveys and the Time Domain Astronomy, with the resulting Petascale data streams and the need for a real-time processing, classification, and decision making. A broad variety of classification and clustering methods have been applied for these tasks, and this remains a very active area of research. Over the past decade we have seen an exponential growth of the astronomical literature involving a variety of ML/AI applications of an ever increasing complexity and sophistication. ML and AI are now a standard part of the astronomical toolkit. As the data complexity continues to increase, we anticipate further advances leading towards a collaborative human-AI discovery.
We introduce a generative model for protein backbone design utilizing geometric products and higher order message passing. In particular, we propose Clifford Frame Attention (CFA), an extension of the invariant point attention (IPA) architecture from AlphaFold2, in which the backbone residue frames and geometric features are represented in the projective geometric algebra. This enables to construct geometrically expressive messages between residues, including higher order terms, using the bilinear operations of the algebra. We evaluate our architecture by incorporating it into the framework of FrameFlow, a state-of-the-art flow matching model for protein backbone generation. The proposed model achieves high designability, diversity and novelty, while also sampling protein backbones that follow the statistical distribution of secondary structure elements found in naturally occurring proteins, a property so far only insufficiently achieved by many state-of-the-art generative models.
We apply pre-trained Vision Transformers (ViTs), originally developed for image recognition, to the analysis of astronomical spectral data. By converting traditional one-dimensional spectra into two-dimensional image representations, we enable ViTs to capture both local and global spectral features through spatial self-attention. We fine-tune a ViT pretrained on ImageNet using millions of spectra from the SDSS and LAMOST surveys, represented as spectral plots. Our model is evaluated on key tasks including stellar object classification and redshift (zz) estimation, where it demonstrates strong performance and scalability. We achieve classification accuracy higher than Support Vector Machines and Random Forests, and attain R2R^2 values comparable to AstroCLIP's spectrum encoder, even when generalizing across diverse object types. These results demonstrate the effectiveness of using pretrained vision models for spectroscopic data analysis. To our knowledge, this is the first application of ViTs to large-scale, which also leverages real spectroscopic data and does not rely on synthetic inputs.
The dominant mechanism forming multiple stellar systems in the high-mass regime (M_\ast \gtrsim 8 MM_{\odot}) remained unknown because direct imaging of multiple protostellar systems at early phases of high-mass star formation is very challenging. High-mass stars are expected to form in clustered environments containing binaries and higher-order multiplicity systems. So far only a few high-mass protobinary systems, and no definitive higher-order multiples, have been detected. Here we report the discovery of one quintuple, one quadruple, one triple and four binary protostellar systems simultaneously forming in a single high-mass protocluster, G333.23--0.06, using Atacama Large Millimeter/submillimeter Array high-resolution observations. We present a new example of a group of gravitationally bound binary and higher-order multiples during their early formation phases in a protocluster. This provides the clearest direct measurement of the initial configuration of primordial high-order multiple systems, with implications for the in situ multiplicity and its origin. We find that the binary and higher-order multiple systems, and their parent cores, show no obvious sign of disk-like kinematic structure. We conclude that the observed fragmentation into binary and higher-order multiple systems can be explained by core fragmentation, indicating its crucial role in establishing the multiplicity during high-mass star cluster formation.
[abridged] We present a detailed comparison of fundamental dark matter halo properties retrieved by a substantial number of different halo finders. These codes span a wide range of techniques including friends-of-friends (FOF), spherical-overdensity (SO) and phase-space based algorithms. We further introduce a robust (and publicly available) suite of test scenarios that allows halo finder developers to compare the performance of their codes against those presented here. This set includes mock haloes containing various levels and distributions of substructure at a range of resolutions as well as a cosmological simulation of the large-scale structure of the universe. All the halo finding codes tested could successfully recover the spatial location of our mock haloes. They further returned lists of particles (potentially) belonging to the object that led to coinciding values for the maximum of the circular velocity profile and the radius where it is reached. All the finders based in configuration space struggled to recover substructure that was located close to the centre of the host halo and the radial dependence of the mass recovered varies from finder to finder. Those finders based in phase space could resolve central substructure although they found difficulties in accurately recovering its properties. Via a resolution study we found that most of the finders could not reliably recover substructure containing fewer than 30-40 particles. However, also here the phase space finders excelled by resolving substructure down to 10-20 particles. By comparing the halo finders using a high resolution cosmological volume we found that they agree remarkably well on fundamental properties of astrophysical significance (e.g. mass, position, velocity, and peak of the rotation curve).
The ongoing discrepancy in the Hubble constant (H0H_0) estimates obtained through local distance ladder methods and early universe observations poses a significant challenge to the Λ\LambdaCDM model, suggesting potential new physics. Type II supernovae (SNe II) offer a promising technique for determining H0H_0 in the local universe independently of the traditional distance ladder approach, opening up a complimentary path for testing this discrepancy. We aim to provide the first H0H_0 estimate using the tailored expanding photosphere method (EPM) applied to SNe II, made possible by recent advancements in spectral modelling that enhance its precision and efficiency. Our tailored EPM measurement utilizes a spectral emulator to interpolate between radiative transfer models calculated with TARDIS, allowing us to fit supernova spectra efficiently and derive self-consistent values for luminosity-related parameters. We apply the method on public data for ten SNe II at redshifts between 0.01 and 0.04. Our analysis demonstrates that the tailored EPM allows for H0H_0 measurements with precision comparable to the most competitive established techniques, even when applied to literature data not designed for cosmological applications. We find an independent H0H_0 value of 74.9±1.974.9\pm1.9 (stat) km/s/Mpc, which is consistent with most current local measurements. Considering dominant sources of systematic effects, we conclude that our systematic uncertainty is comparable to or less than the current statistical uncertainty. This proof-of-principle study highlights the potential of the tailored EPM as a robust and precise tool for investigating the Hubble tension independently of the local distance ladder. Observations of SNe II tailored to H0H_0 estimation can make this an even more powerful tool by improving the precision and by allowing us to better understand and control systematic uncertainties.
Most major discoveries in astronomy have come from unplanned discoveries made by surveying the Universe in a new way, rather than by testing a hypothesis or conducting an investigation with planned outcomes. Next generation radio continuum surveys such as the Evolutionary Map of the Universe (EMU: the radio continuum survey on the new Australian SKA Pathfinder telescope), will significantly expand the volume of observational phase space, so we can be reasonably confident that we will stumble across unexpected new phenomena or new types of object. However, the complexity of the instrument and the large data volumes mean that it may be non-trivial to identify them. On the other hand, if we don't, then we may be missing out on the most exciting science results from EMU. We have therefore started a project called "WTF", which explicitly aims to mine EMU data to discover unexpected science that is not part of our primary science goals, using a variety of machine-learning techniques and algorithms. Although targeted specifically at EMU, we expect this approach will have broad applicability to astronomical survey data.
Binary neutron star (BNS) post-merger gravitational-wave emission can occur in the aftermath of a BNS merger -- provided the system avoids prompt collapse to a black hole -- as a quasistable hypermassive remnant experiences quadrupolar oscillations and non-axisymmetric deformations. The post-merger gravitational-wave spectrum possesses a characteristic peak frequency that has been shown to be dependent on the binary chirp mass and the neutron star equation of state (EoS), rendering post-merger gravitational waves a powerful tool for constraining neutron star composition. Unfortunately, the BNS post-merger signal is emitted at high (1.5\gtrsim 1.5 kHz) frequencies, where ground-based gravitational wave detectors suffer from reduced sensitivity. It is therefore unlikely that post-merger signals will be detected with sufficient signal-to-noise ratio (SNR) until the advent of next-generation detectors. However, by employing empirical relations derived from numerical relativity simulations, we can combine information across an ensemble of BNS mergers, allowing us to obtain EoS constraints with many low-SNR signals. We present a hierarchical Bayesian method for deriving constraints on R1.6R_{1.6}, the radius of a 1.6M\mathrm{M_{\odot}} neutron star, through an ensemble analysis of binary neutron star mergers. We apply this method to simulations of the next two LIGO-Virgo-KAGRA observing runs, O4 and O5, as well as an extended 4-year run at A+ sensitivity, demonstrating the potential of our approach to yield EoS information from the post-merger signal with current-generation detectors. The A+ 4-year scenario is predicted to improve the constraint on R1.6R_{1.6} from the currently available multimessenger-based 95\% credible interval (C.I.) uncertainty of R1.6=12.070.77+0.98R_{1.6}=12.07^{+0.98}_{-0.77} km to R1.6=11.910.56+0.80R_{1.6}=11.91^{+0.80}_{-0.56} km, a 22% reduction of the 95% C.I. width.
In the third APOKASC catalog, we present data for the complete sample of 15,808 evolved stars with APOGEE spectroscopic parameters and Kepler asteroseismology. We used ten independent asteroseismic analysis techniques and anchor our system on fundamental radii derived from Gaia LL and spectroscopic TeffT_{\rm eff}. We provide evolutionary state, asteroseismic surface gravity, mass, radius, age, and the spectroscopic and asteroseismic measurements used to derive them for 12,418 stars. This includes 10,036 exceptionally precise measurements, with median fractional uncertainties in \nmax, \dnu, mass, radius and age of 0.6\%, 0.6\%, 3.8\%, 1.8\%, and 11.1\% respectively. We provide more limited data for 1,624 additional stars which either have lower quality data or are outside of our primary calibration domain. Using lower red giant branch (RGB) stars, we find a median age for the chemical thick disk of 9.14±0.05(ran)±0.9(sys)9.14 \pm 0.05 ({\rm ran}) \pm 0.9 ({\rm sys}) Gyr with an age dispersion of 1.1 Gyr, consistent with our error model. We calibrate our red clump (RC) mass loss to derive an age consistent with the lower RGB and provide asymptotic GB and RGB ages for luminous stars. We also find a sharp upper age boundary in the chemical thin disk. We find that scaling relations are precise and accurate on the lower RGB and RC, but they become more model dependent for more luminous giants and break down at the tip of the RGB. We recommend the usage of multiple methods, calibration to a fundamental scale, and the usage of stellar models to interpret frequency spacings.
Foundational models have emerged as a powerful paradigm in deep learning field, leveraging their capacity to learn robust representations from large-scale datasets and effectively to diverse downstream applications such as classification. In this paper, we present Astromer 2 a foundational model specifically designed for extracting light curve embeddings. We introduce Astromer 2 as an enhanced iteration of our self-supervised model for light curve analysis. This paper highlights the advantages of its pre-trained embeddings, compares its performance with that of its predecessor, Astromer 1, and provides a detailed empirical analysis of its capabilities, offering deeper insights into the model's representations. Astromer 2 is pretrained on 1.5 million single-band light curves from the MACHO survey using a self-supervised learning task that predicts randomly masked observations within sequences. Fine-tuning on a smaller labeled dataset allows us to assess its performance in classification tasks. The quality of the embeddings is measured by the F1 score of an MLP classifier trained on Astromer-generated embeddings. Our results demonstrate that Astromer 2 significantly outperforms Astromer 1 across all evaluated scenarios, including limited datasets of 20, 100, and 500 samples per class. The use of weighted per-sample embeddings, which integrate intermediate representations from Astromer's attention blocks, is particularly impactful. Notably, Astromer 2 achieves a 15% improvement in F1 score on the ATLAS dataset compared to prior models, showcasing robust generalization to new datasets. This enhanced performance, especially with minimal labeled data, underscores the potential of Astromer 2 for more efficient and scalable light curve analysis.
Among the many challenges posed by the huge data volumes produced by the new generation of astronomical instruments there is also the search for rare and peculiar objects. Unsupervised outlier detection algorithms may provide a viable solution. In this work we compare the performances of six methods: the Local Outlier Factor, Isolation Forest, k-means clustering, a measure of novelty, and both a normal and a convolutional autoencoder. These methods were applied to data extracted from SDSS stripe 82. After discussing the sensitivity of each method to its own set of hyperparameters, we combine the results from each method to rank the objects and produce a final list of outliers.
There are no more papers matching your filters at the moment.