ML Arxiv Haul #7
Just last month I made my last Arxiv haul post with most of the papers which I’d seen on Twitter, Reddit, and wherever else. More papers have stacked up, and I’ve decided to do a summarize ~half of them now rather than keep falling behind.
The paper explores adversarial examples in brain scans — very unlike anything else which I’ve seen in the ML world. When you look at the images you can still tell what they are, but the authors have evidence that your neurons get a bit of a workout in figuring it out. I could have sworn it was a bit (annoying? uncomfortable?) to scan through the example images at first.
Specialized library for health ML, where there is some obfuscation / censorship around patients’ deaths.
A collaborative paper from Masakhane (African NLP community) about translation between 16 languages, including language pairs without parallel corpora. The team discusses challenges for low-resource languages, and creates T5 and ByT5 models.
This goes a bit too deep into ‘causal ML’ diagrams, but this is a possible framework for seeing if models are ‘predicting’ an outcome or finding a person with a similar demographic info in the training dataset.
This is my first co-authorship credit! In spring 2021, Google opened a repo to accept voluntarily contributed benchmarks — an opportunity to get onto a major paper and have your task run against new non-public LLMs. It’s been pretty cool seeing my tasks (disambiguation_qs, which_wiki_edit) pop up or get directly discussed in Google and DeepMind papers. It looks like there’s a variety of approaches to taking subsets of tasks, but it isn’t yet visibly being picked up by OpenAI, Microsoft, AllenAI, etc…. I did a presentation at work which I should turn into a blog post here soon.
Side note about citations — when this launched I joked about citing it as Doiron et al., and everyone got named in a citation in Long Range Language Modeling via Gated State Spaces (weirdly Nick → Nicholas there). Hundreds of authors is an oddity in CS, but common in particle physics and genetics, so you can just include the title and a few of the first listed authors if you like.
Experiments in removing (pruning) training data from Imagenet without impacting accuracy.
I’ve been looking into this space because the European NLP people said a major concern is summarizing and/or simplifying materials for language learners. They fine-tune mT5 and mBART to do a sort of translation plus formality change, with some unclear work to include zero-shot experiments. mT5 includes many languages so this could be useful to explore.
When I was at the Probabilistic AI course in Finland, everyone was excited about generating content with diffusion models. HuggingFace, which already has the popular transformers model, is looking for another hit with a diffusers repo.
Work on text summarization which preserves factual accuracy.
When we run an NLG model, there’s a choice about whether each next token is picked from the highest probability, sampling of probabilities, or a variety of somewhat-more-complex decoding methods. On WinoMT, which measures gender bias in translations, this beam search method improves performance.
A dataset to see if language models can predict actual events. I would say that you need to take care that your pretrained model is itself older than the events (the researchers use GPT-2 and T5). For prompting, the model gets one article with information available at prediction time. Overall this is a little interesting but frustrated by the small size of the models for 2022, and not so much focus on difficulty of a prediction (such as collecting numbers from a prediction market).
This is an early 2022 pre-print, and cites other recent work. I remember seeing Google and Siri being ‘uncomfortable’ about directing people to abortion clinics, or getting tricked by ‘crisis centers’. This paper tracks searches in multiple locations over time, and finds that clinics are usually returned, but that ‘crisis centers’ get better positioning in poorer and more rural areas.
After going through a Probabilistic AI mini-course which had a strong interest in Bayesian methods and Bayesian neural networks, it’s fascinating to see this 2020 Google paper about what aspects of this are accepted or rejected in industry. Discusses a ‘cold posterior’ which goes against Bayesian dogma, and would continue to be discussed in 2021, but is not super popular in research that I could find.
With a subset of ImageNet, the researchers try to predict the amount of data needed to reach a target accuracy. Their final method makes several tests and compares multiple prediction curves as it goes through training, so there’s no one ‘method’ to estimating this.
After a paper went viral for fixing GPT math errors by prompting “Let’s think step by step”, this paper is pitched as some type of study or taxonomy of this approach, putting forth the name “language model cascades”. The text gets a little muddled and I wonder if this paper initially sought to show something else? In any case it looks like a good method.
Asking language models to put a number on their own accuracy. Straightforward and useful.
Interesting dataset which the authors have framed as a summarization problem that matters. Expert summaries of very long legal documents. Includes different granularities of summarization.
For 256 GB of just legal documents, see Pile of Law.
Facebook/Meta’s work on updating factual associations inside of language model knowledge led to benchmarks LAMA and mLAMA. Salesforce proposes a trainable layer between the token/embedding layer and the rest of the network, which I guess is shifting specific tokens to update those answers? It’s tricky to say without tinkering whether this is just swapping tokens around for specific questions (i.e. “current US president” returning an updated name, but “characteristics and policy of current US president” concept not changing).
Google has an AI blog post announcing these ‘Plex’ models but sort of hand-waving about what makes them ‘reliable’. The main improvements are work on robustness and in reporting uncertainty. Though there are comparisons to Bayesian neural networks and probabilistic ML, the different approaches are not discussed in detail.
This work references the uncertainty-baselines project which has been developed for the past 2 years.
Using reinforcement learning to tune model responses (i.e. making less-toxic responses), not forgetting information and facts. Still very cool.
A number of major internet companies have been praising NIST’s pick of a few lattice-based cryptography algorithms for classical computers to stay secure in the post-quantum era. Here Facebook/Meta team applies ML to the task of decrypting information (itself a pretty big task) by pointing it at this less-tested algorithm, starting with a small key size and scaling up. To clarify, they are exploring whether ML on a classical computer is likely to learn to decrypt the system.
Researchers paste a cartoon gorilla into lung cancer images, referencing previous experiments in change blindness and limited observation skills. Most experts did not report the gorilla, but eye-tracking shows that they did fixate on it as an anomaly:
of the 20 radiologists who did not report the gorilla, 12 looked directly at the gorilla’s location when it was visible. The mean dwell time on the gorilla amongst this group was 547ms
Towards Robust Spanish Author Profiling and Lessons Learned from Adversarial Attacks
This paper popped up in my e-mail because it uses my seq2seq model for Spanish gender-reinflection.
Author profiling accuracy drops when you break tokenization with invisible characters (tbh not great work if someone pre-processes their text). When you use my seq2seq model (labeled Counterfactual here) the author profiling-by-gender accuracy drops from 0.738 to 0.515 (almost doubling the error).
I took a look into other toxic language models after the ‘GPT-4chan’ debacle… were other models in the HuggingFace / NLP ecosystem also full of toxic text? ToxiGen is a well-designed dataset which instead has a mix of in-the-wild, adversarially-generated, and human-in-the-loop processes to build up a large dataset of toxic texts.
I’m not super interested in the paper or problem, but this is the first time that I’m seeing this ‘Right-for-the-Right-Reason metrics’ (RRR) term in explainable AI, which is descriptive and necessary.
It goes back to an IJCAI-17 paper with different authors at another university, but the term does not get used often.
In the AI + cybersecurity world, there are a variety of theories about how attackers will approach ML systems. This paper does a survey of methods to either fill the training data space with misleading examples, or craft examples which in training effectively build a backdoor/shortcut to override intentional features (i.e. I associate a kid-safe social media account with unique patterns or phrases which then will pass through content filters). They also discuss defenses by ‘sanitizing’ data or analyzing the model.
Long reads / Overview docs
(lifelong learning = continually-trained models)
AI legend Yann LeCun describes his vision for the future of ML, posts it on OpenReview for public comment. Controversial response by Schmidhuber (who frequently asks LeCun and others to cite his early work in ML as the original). I swear there was a controversy about ethics not being emphasized enough here, and two AI ethics critics being the first commenters, but after digging I must have mixed it up with something else?
