Accurate ground truth estimation in medical screening programs often relies on coalitions of experts and peer second opinions. Algorithms that efficiently aggregate noisy annotations can enhance screening workflows, particularly when data arrive continuously and expert proficiency is initially unknown. However, existing algorithms do not meet the requirements for seamless integration into screening pipelines. We therefore propose an adaptive approach for real-time annotation that (I) supports on-the-fly labeling of incoming data, (II) operates without prior knowledge of medical experts or pre-labeled data, and (III) dynamically queries additional experts based on the latent difficulty of each instance. The method incrementally gathers expert opinions until a confidence threshold is met, providing accurate labels with reduced annotation overhead. We evaluate our approach on three multi-annotator classification datasets across different modalities. Results show that our adaptive querying strategy reduces the number of expert queries by up to 50% while achieving accuracy comparable to a non-adaptive baseline. Our code is available at this https URL

Extracting from shared resources requires making choices to balance personal profit and sustainability. We present the results of a behavioural experiment wherein we manipulate the default extraction from a finite resource. Participants were exposed to two treatments -- pro-social or self-serving extraction defaults -- and a control without defaults. We examined the persistence of these nudges by removing the default after five rounds. Results reveal that a self-serving default increased the average extraction while present, whereas a pro-social default only decreased extraction for the first two rounds. Notably, the influence of defaults depended on individual inclinations, with cooperative individuals extracting more under a self-serving default, and selfish individuals less under a pro-social default. After the removal of the default, we observed no significant differences with the control treatment. Our research highlights the potential of defaults as cost-effective tools for promoting sustainability, while also advocating for a careful use to avoid adverse effects.