Transformer-based large language models (LLMs) excel in modeling complex language patterns but face significant computational costs during inference, especially with long inputs due to the attention mechanism's memory overhead. We observe that certain attention heads exhibit a distribution where the attention weights concentrate on tokens near the query token, termed as recency aware, which focuses on local and short-range dependencies. Leveraging this insight, we propose RecurFormer, a novel architecture that replaces these attention heads with linear recurrent neural networks (RNNs), specifically the Mamba architecture. This replacement reduces the cache size without evicting tokens, thus maintaining generation quality. RecurFormer retains the ability to model long-range dependencies through the remaining attention heads and allows for reusing pre-trained Transformer-based LLMs weights with continual training. Experiments demonstrate that RecurFormer matches the original model's performance while significantly enhancing inference efficiency. Our approach provides a practical solution to the computational challenges of Transformer-based LLMs inference, making it highly attractive for tasks involving long inputs.

Variable selection is a challenging problem in high-dimensional sparse learning, especially when group structures exist. Group SLOPE performs well for the adaptive selection of groups of predictors. However, the block non-separable group effects in Group SLOPE make existing methods either invalid or inefficient. Consequently, Group SLOPE tends to incur significant computational costs and memory usage in practical high-dimensional scenarios. To overcome this issue, we introduce a safe screening rule tailored for the Group SLOPE model, which efficiently identifies inactive groups with zero coefficients by addressing the block non-separable group effects. By excluding these inactive groups during training, we achieve considerable gains in computational efficiency and memory usage. Importantly, the proposed screening rule can be seamlessly integrated into existing solvers for both batch and stochastic algorithms. Theoretically, we establish that our screening rule can be safely employed with existing optimization algorithms, ensuring the same results as the original approaches. Experimental results confirm that our method effectively detects inactive feature groups and significantly boosts computational efficiency without compromising accuracy.

We report new measurements of the ratio of the electric form factor to the magnetic form factor of the neutron, GEn/GMn, obtained via recoil polarimetry from the quasielastic ^2H(vec{e},e'vec{n})^1H reaction at Q^2 values of 0.45, 1.13, and 1.45 (GeV/c)^2 with relative statistical uncertainties of 7.6 and 8.4% at the two higher Q^2 points, which were not reached previously via polarization measurements. Scale and systematic uncertainties are small.

We report values for the neutron electric to magnetic form factor ratio, GEn/GMn, deduced from measurements of the neutron's recoil polarization in the quasielastic 2H(\vec{e},e'\vec{n})1H reaction, at three Q^2 values of 0.45, 1.13, and 1.45 (GeV/c)^2. The data at Q^2 = 1.13 and 1.45 (GeV/c)^2 are the first direct experimental measurements of GEn employing polarization degrees of freedom in the Q^2 > 1 (GeV/c)^2 region and stand as the most precise determinations of GEn for all values of Q^2.

The past decade has witnessed a dramatic acceleration of lattice quantum chromodynamics calculations in nuclear and particle physics. This has been due to both significant progress in accelerating the iterative linear solvers using multi-grid algorithms, and due to the throughput improvements brought by GPUs. Deploying hierarchical algorithms optimally on GPUs is non-trivial owing to the lack of parallelism on the coarse grids, and as such, these advances have not proved multiplicative. Using the QUDA library, we demonstrate that by exposing all sources of parallelism that the underlying stencil problem possesses, and through appropriate mapping of this parallelism to the GPU architecture, we can achieve high efficiency even for the coarsest of grids. Results are presented for the Wilson-Clover discretization, where we demonstrate up to 10x speedup over present state-of-the-art GPU-accelerated methods on Titan. Finally, we look to the future, and consider the software implications of our findings.

Recurrent neural networks (RNNs) based automatic speech recognition has nowadays become prevalent on mobile devices such as smart phones. However, previous RNN compression techniques either suffer from hardware performance overhead due to irregularity or significant accuracy loss due to the preserved regularity for hardware friendliness. In this work, we propose RTMobile that leverages both a novel block-based pruning approach and compiler optimizations to accelerate RNN inference on mobile devices. Our proposed RTMobile is the first work that can achieve real-time RNN inference on mobile platforms. Experimental results demonstrate that RTMobile can significantly outperform existing RNN hardware acceleration methods in terms of inference accuracy and time. Compared with prior work on FPGA, RTMobile using Adreno 640 embedded GPU on GRU can improve the energy-efficiency by about 40$\times$ while maintaining the same inference time.

Weight pruning of deep neural networks (DNNs) has been proposed to satisfy the limited storage and computing capability of mobile edge devices. However, previous pruning methods mainly focus on reducing the model size and/or improving performance without considering the privacy of user data. To mitigate this concern, we propose a privacy-preserving-oriented pruning and mobile acceleration framework that does not require the private training dataset. At the algorithm level of the proposed framework, a systematic weight pruning technique based on the alternating direction method of multipliers (ADMM) is designed to iteratively solve the pattern-based pruning problem for each layer with randomly generated synthetic data. In addition, corresponding optimizations at the compiler level are leveraged for inference accelerations on devices. With the proposed framework, users could avoid the time-consuming pruning process for non-experts and directly benefit from compressed models. Experimental results show that the proposed framework outperforms three state-of-art end-to-end DNN frameworks, i.e., TensorFlow-Lite, TVM, and MNN, with speedup up to 4.2X, 2.5X, and 2.0X, respectively, with almost no accuracy loss, while preserving data privacy.

We report the first measurement of the \eep three-body breakup reaction cross sections in helium-3 ($^3$He) and tritium ($^3$H) at large momentum transfer ($\langle Q^2 \rangle \approx 1.9$ (GeV/c)$^2$) and $x_B>1$ kinematics, where the cross section should be sensitive to quasielastic (QE) scattering from single nucleons. The data cover missing momenta $40 \le p_{miss} \le 500$ MeV/c that, in the QE limit with no rescattering, equals the initial momentum of the probed nucleon. The measured cross sections are compared with state-of-the-art ab-initio calculations. Overall good agreement, within $\pm20\%$, is observed between data and calculations for the full $p_{miss}$ range for $^3$H and for $100 \le p_{miss} \le 350$ MeV/c for $^3$He. Including the effects of rescattering of the outgoing nucleon improves agreement with the data at $p_{miss} > 250$ MeV/c and suggests contributions from charge-exchange (SCX) rescattering. The isoscalar sum of $^3$He plus $^3$H, which is largely insensitive to SCX, is described by calculations to within the accuracy of the data over the entire $p_{miss}$ range. This validates current models of the ground state of the three-nucleon system up to very high initial nucleon momenta of $500$ MeV/c.

The strong interaction is not well understood at low energy, or for interactions with low momentum transfer $Q^2$, but one of the clearest insights we have comes from Chiral Perturbation Theory ($\chi$PT). This effective treatment gives testable predictions for the nucleonic generalized polarizabilities -- fundamental quantities describing the nucleon's response to an external field. We have measured the proton's generalized spin polarizabilities in the region where $\chi$PT is expected to be valid. Our results include the first ever data for the transverse-longitudinal spin polarizability $\delta_{LT}$, and also extend the coverage of the polarizability $\bar{d_2}$ to very low $Q^2$ for the first time. These results were extracted from moments of the structure function $g_2$, a quantity which characterizes the internal spin structure of the proton. Our experiment ran at Jefferson Lab using a polarized electron beam and a polarized solid ammonia (NH$_3$) target. The $\delta_{LT}$ polarizability has remained a challenging quantity for $\chi$PT to reproduce, despite its reduced sensitivity to higher resonance contributions; recent competing calculations still disagree with each other and also diverge from the measured neutron data at very low $Q^2$. Our proton results provide discriminating power between existing calculations, and will help provide a better understanding of this strong QCD regime.

This paper presents a novel end-to-end approach to program repair based on sequence-to-sequence learning. We devise, implement, and evaluate a system, called SequenceR, for fixing bugs based on sequence-to-sequence learning on source code. This approach uses the copy mechanism to overcome the unlimited vocabulary problem that occurs with big code. Our system is data-driven; we train it on 35,578 samples, carefully curated from commits to open-source repositories. We evaluate it on 4,711 independent real bug fixes, as well on the Defects4J benchmark used in program repair research. SequenceR is able to perfectly predict the fixed line for 950/4711 testing samples, and find correct patches for 14 bugs in Defects4J. It captures a wide range of repair operators without any domain-specific top-down design.

Group Ordered Weighted $L_{1}$-Norm (Group OWL) regularized models have emerged as a useful procedure for high-dimensional sparse multi-task learning with correlated features. Proximal gradient methods are used as standard approaches to solving Group OWL models. However, Group OWL models usually suffer huge computational costs and memory usage when the feature size is large in the high-dimensional scenario. To address this challenge, in this paper, we are the first to propose the safe screening rule for Group OWL models by effectively tackling the structured non-separable penalty, which can quickly identify the inactive features that have zero coefficients across all the tasks. Thus, by removing the inactive features during the training process, we may achieve substantial computational gain and memory savings. More importantly, the proposed screening rule can be directly integrated with the existing solvers both in the batch and stochastic settings. Theoretically, we prove our screening rule is safe and also can be safely applied to the existing iterative optimization algorithms. Our experimental results demonstrate that our screening rule can effectively identify the inactive features and leads to a significant computational speedup without any loss of accuracy.

Convolutional neural networks (CNNs) are becoming increasingly deeper, wider, and non-linear because of the growing demand on prediction accuracy and analysis quality. The wide and deep CNNs, however, require a large amount of computing resources and processing time. Many previous works have studied model pruning to improve inference performance, but little work has been done for effectively reducing training cost. In this paper, we propose ClickTrain: an efficient and accurate end-to-end training and pruning framework for CNNs. Different from the existing pruning-during-training work, ClickTrain provides higher model accuracy and compression ratio via fine-grained architecture-preserving pruning. By leveraging pattern-based pruning with our proposed novel accurate weight importance estimation, dynamic pattern generation and selection, and compiler-assisted computation optimizations, ClickTrain generates highly accurate and fast pruned CNN models for direct deployment without any extra time overhead, compared with the baseline training. ClickTrain also reduces the end-to-end time cost of the pruning-after-training method by up to 2.3X with comparable accuracy and compression ratio. Moreover, compared with the state-of-the-art pruning-during-training approach, ClickTrain provides significant improvements both accuracy and compression ratio on the tested CNN models and datasets, under similar limited training time.

Insider threats (InTs) within organizations are small in number but have a disproportionate ability to damage systems, information, and infrastructure. Existing InT research studies the problem from psychological, technical, and educational perspectives. Proposed theories include research on psychological indicators, machine learning, user behavioral log analysis, and educational methods to teach employees recognition and mitigation techniques. Because InTs are a human problem, training methods that address InT detection from a behavioral perspective are critical. While numerous technological and psychological theories exist on detection, prevention, and mitigation, few training methods prioritize psychological indicators. This literature review studied peer-reviewed, InT research organized by subtopic and extracted critical theories from psychological, technical, and educational disciplines. In doing so, this is the first study to comprehensively organize research across all three approaches in a manner which properly informs the development of an InT education platform.

One of the key steps in quantum algorithms is to prepare an initial quantum superposition state with different kinds of features. These so-called state preparation algorithms are essential to the behavior of quantum algorithms, and complicated state preparation algorithms are difficult to develop correctly and effectively. This paper presents Pqasm: a high-assurance framework implemented with the Coq proof assistant, allowing us to certify our Pqasm tool to correctly reflect quantum program behaviors. The key in the framework is to reduce the program correctness assurance of a program containing a quantum superposition state to the program correctness assurance for the program state without superposition. The reduction allows the development of an effective testing framework for testing quantum state preparation algorithm implementations on a classical computer - considered to be a hard problem with no clear solution until this point. We utilize the QuickChick property-based testing framework to test state preparation programs. We evaluated the effectiveness of our approach over 5 case studies implemented using Pqasm; such cases are not even simulatable in the current quantum simulators.

We address the calculation of dynamical correlation functions for many fermion systems at zero temperature, using the auxiliary-field quantum Monte Carlo method. The two-dimensional Hubbard hamiltonian is used as a model system. Although most of the calculations performed here are for cases where the sign problem is absent, the discussions are kept general for applications to physical problems when the sign problem does arise. We study the use of twisted boundary conditions to improve the extrapolation of the results to the thermodynamic limit. A strategy is proposed to drastically reduce finite size effects relying on a minimization among the twist angles. This approach is demonstrated by computing the charge gap at half-filling. We obtain accurate results showing the scaling of the gap with the interaction strength $U$, connecting to the scaling of the unrestricted Hartree-Fock method at small $U$ and Bethe Ansatz exact result in one dimension at large $U$. A new algorithm is then proposed to compute dynamical Green functions and correlation functions which explicitly varies the number of particles during the random walks in the manifold of Slater determinants. In dilute systems, such as ultracold Fermi gases, this algorithm enables calculations with much more favorable complexity, with computational cost proportional to basis size or the number of lattice sites.

Let $$A=\left(\begin{array}{cc} H_1 & E^*\\ E & H_2\end{array}\right) \quad \hbox{ and } \quad \wtd A=\left(\begin{array}{cc} H_1 & O\\ O & H_2\end{array}\right)$$ be two $N$-by-$N$ Hermitian matrices with eigenvalues $\lambda_1 \ge \cdots \ge \lambda_{N}$ and $\wtd \lambda_1 \ge \cdots \ge \wtd \lambda_N$, respectively. \iffalse There are two kinds of perturbation bounds on $|\lambda_i - \wtd \lambda_i|$: $|\lambda_i- \wtd \lambda_i| \le \|E\|$, where $\|E\|$ is the largest singular value of $\|E\|$, regardless of $H_i$'s spectral distributions, and $|\lambda_i - \wtd \lambda_i| \le \|E\|^2/\eta$, where $\eta$ is the minimum gap between $H_i$'s spectra. \end{enumerate} Bounds of the first kind overestimate the changes when $\|E\|\ll\eta$ while those of the second kind may blow up when $\eta$ is too tiny. \fi Denote by $\|E\|$ the spectral norm of the matrix $E$, and $\eta$ the spectral gap between the spectra of $H_1$ and $H_2$. It is shown that $$|\lambda_i - \wtd \lambda_i| \le {2\|E\|^2 \over \eta+\sqrt{\eta^2+4\|E\|^2}} \, ,$$ which improves all the existing results. Similar bounds are obtained for singular values of matrices under block perturbations.

We formulate a quantum Monte Carlo (QMC) method for calculating the ground state of many-boson systems. The method is based on a field-theoretical approach, and is closely related to existing fermion auxiliary-field QMC methods which are applied in several fields of physics. The ground-state projection is implemented as a branching random walk in the space of permanents consisting of identical single-particle orbitals. The method is in principle exact. We illustrate this method with a trapped atomic boson gas, where the atoms interact via an attractive or repulsive contact two-body potential. We choose as the single-particle basis a real-space grid. We compare with exact results in small systems, and arbitrarily-sized systems of untrapped bosons with attractive interactions in one dimension, where analytical solutions exist. We also compare with the corresponding Gross-Pitaevskii (GP) mean-field calculations for trapped atoms, and discuss the close formal relation between our method and the GP approach. Our method provides a way to systematically improve upon GP while using the same framework, capturing interaction and correlation effects with a stochastic, coherent ensemble of non-interacting solutions. We discuss various algorithmic issues, including importance sampling and the back-propagation technique for computing observables, and illustrate them with numerical studies. We show results for systems with up to N ~ 400 bosons.

Policy specification for personal user data is a hard problem, as it depends on many factors that cannot be predetermined by system developers. Simultaneously, systems are increasingly relying on users to make security decisions. In this paper, we propose the approach of Policy by Example (PyBE) for specifying user-specific security policies. PyBE brings the benefits of the successful approach of programming by example (PBE) for program synthesis to the policy specification domain. In PyBE, users provide policy examples that specify if actions should be allowed or denied in certain scenarios. PyBE then predicts policy decisions for new scenarios. A key aspect of PyBE is its use of active learning to enable users to correct potential errors in their policy specification. To evaluate PyBE's effectiveness, we perform a feasibility study with expert users. Our study demonstrates that PyBE correctly predicts policies with 76% accuracy across all users, a significant improvement over naive approaches. Finally, we investigate the causes of inaccurate predictions to motivate directions for future research in this promising new domain.

Given a positive integer k, it is natural to ask for a formula for the distance between a given density matrix (i.e., mixed quantum state) and the set of density matrices of rank at most k. This problem has already been solved when "distance" is measured in the trace or Frobenius norm. We solve it for all other unitary similarity invariant norms. We also present some consequences of our formula. For example, in the trace and Frobenius norms, the density matrix that is farthest from the set of low-rank density matrices is the maximally mixed state, but this is not true in many other unitary similarity invariant norms.

Recent progress in Lorentz-covariant quantum field theories of the nuclear many-body problem (quantum hadrodynamics, or QHD) is discussed. The importance of modern perspectives in effective field theory and density functional theory for understanding the successes of QHD is emphasized. To appear in: 150 Years of Quantum Many-Body Theory: A conference in honour of the 65th birthdays of John W. Clark, Alpo J. Kallio, Manfred L. Ristig, and Sergio Rosati.