🔄 mapLimit: Controlled Concurrency in JavaScript
Category: js / utils
Difficulty: medium
When processing large datasets in JavaScript, we often need to perform asynchronous operations on multiple items. However, launching all operations simultaneously can overwhelm system resources or hit API rate limits. This is where the mapLimit function becomes invaluable. What is mapLimit? mapLimit is a concurrency control pattern that applies an asynchronous function to an array of items while limiting how many operations run in parallel. It maintains the order of results to match the input array, regardless of when each operation completes. Key Concepts Concurrency Control Concurrency refers to multiple operations running simultaneously. With mapLimit, we specify exactly how many operations can run at once - not too many (overwhelming resources) and not too few (inefficient processing). Promise Management JavaScript promises are the foundation of this pattern. We track: Active operations count Pending operations queue Results array (preserving original order) Queue Processing As operations complete, new ones begin until all items are processed. This creates a continuous flow without exceeding the concurrency limit. Implementation Breakdown ```javascript async function mapLimit(items, limit, fn) { const promises = []; // Track all generated promises const results = Array(items.length); // Pre-allocate results array let activeCount = 0; // Track active operations let index = 0; // Next item to process const processNext = async () => { const currentIndex = index++; if (currentIndex >= items.length) { return; // No more items to process } activeCount++; try { // Process item and store result at original position results[currentIndex] = await fn(items[currentIndex], currentIndex, items); } catch (error) { // Preserve errors in results array results[currentIndex] = Promise.reject(error); } finally { activeCount--; promises.push(processNext()); // Process next item when this one finishes } }; // Initialize with batch of promises up to the limit while (activeCount < limi...