Open Source at Bloomberg: 1H 2023 Engagement

Originally published on bloomberg.com

Many Bloomberg engineers are active consumers, community participants, contributors, and leaders of various open source projects. Over the first half of 2023, Bloomberg’s Open Source Program Office (OSPO) and its engineers have come together to build, maintain, develop, and fund open source projects. These activities range from filing new commits, contributing pull requests, developing and publishing tools and features, being appointed to and/or sponsoring leadership roles, launching the Bloomberg FOSS (Free and Open Source Software) Contributor Fund, and more. The Bloomberg team also regularly promotes its use cases for these projects and its efforts to advance open source technologies through numerous conference presentations, podcast interviews, and media engagements.

Our long track record of embracing open source is an investment in the future of these technologies. Our goal: to keep the open source ecosystem secure, resilient, and healthy as more and more users and companies rely on open source to keep their systems running.

Here is a roundup of Bloomberg’s open source engagement during the first half of 2023.

Contributions and Projects

• Ceph: Bloomberg’s Distributed Storage team had its first major commit accepted in upstream Ceph; their PR fixed a major bug with race condition in multisite during full sync involving deletes.
• Hera: A new version (v5) of Dyno’s open source project Hera was released in collaboration with Bloomberg and AI Software Engineer Sambhav Kothari. This release expanded Hera to feature parity with workflows and events.
• Lexical: Several Bloomberg engineers have been contributing to this popular JavaScript Text Editor Framework that powers the Bloomberg Terminal. Between them, Attila Novak, Ben Carleton, Ebad Salehi, Ivaylo Pavlov, and Shubhanker Srivastava have landed 20 PRs in the first half of 2023 to improve core functionality.
• Memray: Memray 1.8.0 was published, and it includes a groundbreaking feature: temporal flame graphs. These provide a new perspective on memory usage in programs with dynamic time range analysis. No more static snapshots; explore memory allocation over time!
• MiniLMv2: Bloomberg’s AI researchers published “MiniLMv2.BB,” an open source implementation of the MiniLMv2 method detailed in the ACL 2021 paper “MiniLMv2: Multi-Head Self-Attention Relation Distillation for Compressing Pretrained Transformers.” Using this code, one can distill NLP language models, including BERT or RoBERTa, to a smaller size in order to enable lower latency inference.
• Network Transport Framework (NTF): Bloomberg’s engineers published an open source version of its NTF on GitHub. This collection of asynchronous networking APIs/libraries is designed for sending and receiving data between processes in scalable, high-performance applications.
• PyStack: Bloomberg’s Python Infrastructure team published PyStack, an open source debugging tool that can inspect the stack frames of a running Python process or core file to determine what is happening without having to understand CPython internals. [read more about PyStack]
• TypeScript: Bloomberg’s JavaScript Infrastructure team continues to collaborate with the core TypeScript team to improve the language and tooling. Chi Leung helped modernize namespaces and Leo Elmecker-Plakolm introduced types for an ES2023 feature. There is also exciting progress on experiments to speed up type-checking via parallel builds.
• Ecma TC39: Bloomberg contributes to governance, maintenance, and technical advancement of the JavaScript language. Daniel Ehrenberg is Ecma Vice President and Rob Palmer is Co-Chair of TC39. Software Engineer Ashley Claymore introduced the Await Dictionary proposal. Peter Klecha advanced the Promise.withResolvers() proposal to Stage 3. The Symbols as WeakMap keys and Change Array by Copy features are part of ES2023 thanks to Bloomberg engineers.
• V8: Bloomberg’s ongoing collaboration with Igalia has led to significant wins for the developer experience via contributions to Google’s V8 JavaScript engine. Memory debugging via Heap Snapshots was optimized to be 2x-100x faster. Advanced native profiling on Windows using ETW tracing was enabled to show both native and JavaScript function names.

Community Engagement

• Joined the Adoptium Working Group as an Enterprise Member (and also became a Contributing Member of the Eclipse Foundation) [read more].
• Hosted Ceph Days NYC at its 120 Park Avenue office, where Nathan Hoad of our Storage Engineering team talked about why Bloomberg built a “Message-Driven Telemetry System At Scale” for our Ceph clusters. Our Head of Storage Engineering Matthew Leonard offered the opening remarks. [watch videos of all the Ceph Day NYC talks]
• Sponsored the NYC++ Meetup, which provides an opportunity for the area’s C++ enthusiasts to socialize, network, and learn together.
• Launched a new FOSS Contributor Fund, through a partnership between Bloomberg’s OSPO and Corporate Philanthropy teams, to support the people and communities who maintain open source projects. The fund will run a quarterly election to award up to three grants of $10,000 during each voting cycle. In Q1 2023, the first three projects to receive grants from the Fund were Apache Arrow, Celery, and Curl. The Q2 2023 grant recipients are MkDocs, OpenSSH, and PostgreSQL.
• Sponsored the Python Software Foundation’s hiring of a full-time Deputy Developer-in-Residence to accelerate the velocity of the CPython core dev team. [learn more about the role]
• Pradyun Gedam, an engineer with our Python Infrastructure team, was promoted to CPython core dev. He’s been active in the Python packaging ecosystem since 2017, and works on pip, PyPI, PyPA, Sphinx, TOML, and more. [learn more about Pradyun on Mouse Vs. Python]

Conference Sightings

Watch some of the many talks Bloomberg’s engineers delivered at conferences around the globe in the first half of 2023.

ACCU Conference 2023

• Removing Needless Undefined Behavior for a Safer C++ | Alisdair Meredith
• Managing External API’s in Enterprise Systems | Peter Muldoon
• Rust for the Recalcitrant C++ Programmer | CB Bailey
• Improving Compilation Times: Tools & Techniques | Vittorio Romeo
• C++ Trivial Relocation Through Time | Mungo Gill
• How Bloomberg uses Behavior-Driven Development (BDD) to Test Distributed Systems | Jacqueline Pan
• Monads in Modern C++ | Georgi Koyrushki & Alistair Fisher
• Introduction to Epoch-Based Memory Reclamation | Jeffrey Mendelsohn
• Function Contracts in Practice using C++ | Rostislav Khlebnikov
• What I Learned from Sockets: Applying the Unix Readiness Model When Composing Concurrent Operations in C++ | Filipp Gelman
• Implementing a C++ Coroutine Task from Scratch | Dietmar Kühl

Berlin Buzzwords 2023

• No Mean Feat: Upgrading a Customized Solr to Upstream Solr | Shikhar Srivastava

Cephalocon 2023

• Ceph Multi-Site at Scale: Bloomberg’s Disaster Recovery Journey | Krunal Chheda and Jane Zhu
• Over a Billion Requests Served per Day: Ensuring Everyone Is Happy with Our Ceph Clusters’ Performance | Nathan Hoad and Alex Wojno
• Day 2 Keynote: The Road to Open Source Ceph: Bloomberg’s Enterprise Ceph Journey | Matthew Leonard
• Embracing Ceph: Key Factors Driving Companies to Adopt Ceph Storage Solutions (Panel) | Matthew Leonard

KubeCon + CloudNativeCon Europe 2023

• Customizing Your Buildpacks Build – Yes You Can! | Bloomberg’s Aidan Delaney and VMWare’s Natalie Arellano
• The State and Future of Cloud-Native Model Serving | Bloomberg’s Dan Sun and Amazon’s Theofilos Papapanagiotou

Open Source Summit North America 2023

• What Our OSPOs Have Learned About Measuring Project Health | Alyssa Wright

OpenInfra Summit Vancouver 2023

• The Tortoise Beats the Hare: Upgrading the Operating System of the Cloud at Scale | Jin Geng and Tyler Stachecki
• The OpenInfra Foundation named Bloomberg winner of the 2023 Superuser Awards, recognizing its adoption of and contributions to Linux, OpenStack, Ceph, and Kubernetes infrastructure. The Superuser Awards recognize organizations who use open infrastructure to improve their business while also contributing back to the open infrastructure community.

PyCon US 2023

• Automate Documentation with Sphinx & GitHub Actions (Tutorial) | Olga Matoula and Aya Elsayad
• An async Python web framework with a Rust runtime | Sanskar Jethi
• The wheelhouse of horrors | László Kiss Kollár
• How memory profilers work | Pablo Galindo Salgado
• Python & Bloomberg: An Open Source Duo | Mario Corchero, Bernat Gabor, Pradyun Gedam, László Kiss Kollár, Pablo Galindo Salgado, and Matt Wozniski

In The Media

CppCast podcast

• Episode 362: Modules and build systems (Daniel Ruoso)
• Episode 363: Contracts (Joshua Berne)

The Changelog podcast: How companies are sponsoring OSS (Alyssa Wright)

OSPOlogy: How OSPOs Manage Change In Enterprises For Open Source Adoption (Alyssa Wright)

TODO Group | OSPOlogy: OSPOs Fostering FOSS Sustainability Through Effective Collaborative Funding (Alyssa Wright)

Talk Python To Me podcast

• Episode #406: Reimagining Python’s Packaging Workflows (Pradyun Gedam)
• Episode #419: Debugging Python in Production with PyStack (Pablo Galindo Salgado & Matt Wozniski)

TechTarget: Sidecarless eBPF service mesh sparks debate (Andrey Rybka)

The New Stack

• Will JavaScript Become the Most Popular WebAssembly Language? (Daniel Ehrenberg and Rob Palmer)
• New JavaScript features coming in ES2023 (Daniel Ehrenberg and Rob Palmer)
• What’s Next for JavaScript: New Features to Look Forward to (Daniel Ehrenberg and Rob Palmer)
• Beyond Browsers: The Long-term Future of JavaScript Standards (Daniel Ehrenberg and Rob Palmer)