Advanced Function Composition: Building Elegant Data Processing Pipelines

The Art of Composition

In music, individual notes are simple, but when combined in a specific order, they can create beautiful melodies. Function composition is the same—combining simple functions to build powerful and elegant programs.

Function Composition is one of the core ideas of functional programming. It allows us to:

Break down complex problems into simple functions
Build new functionality through composition
Create clear data transformation pipelines
Write reusable and testable code

Basic Composition Patterns

1. Compose and Pipe

In basic functional programming, we've learned about compose and pipe. Let's explore them in depth:

javascript

// Compose: execute from right to left (direction of mathematical function composition)
const compose =
  (...fns) =>
  (value) =>
    fns.reduceRight((acc, fn) => fn(acc), value);

// Pipe: execute from left to right (more intuitive, Unix pipe style)
const pipe =
  (...fns) =>
  (value) =>
    fns.reduce((acc, fn) => fn(acc), value);

// Basic functions
const double = (x) => x * 2;
const increment = (x) => x + 1;
const square = (x) => x * x;

// Using compose: from right to left
const calc1 = compose(
  square, // 3. Finally square
  increment, // 2. Then add 1
  double // 1. First double
);

console.log(calc1(3)); // (3 * 2 + 1)² = 49

// Using pipe: from left to right (more readable)
const calc2 = pipe(
  double, // 1. First double
  increment, // 2. Then add 1
  square // 3. Finally square
);

console.log(calc2(3)); // (3 * 2 + 1)² = 49

2. Composition of Multi-parameter Functions

javascript

// Problem: Only the first function can receive multiple parameters
const add = (a, b) => a + b;
const multiply = (x) => x * 2;
const subtract = (x) => x - 1;

// ❌ This won't work
const bad = pipe(add, multiply, subtract);
// bad(5, 3) => NaN (because multiply and subtract can only receive one parameter)

// ✅ Currying solution
const curry = (fn) => {
  return function curried(...args) {
    if (args.length >= fn.length) {
      return fn(...args);
    }
    return (...nextArgs) => curried(...args, ...nextArgs);
  };
};

const curriedAdd = curry((a, b) => a + b);

const good = pipe(
  curriedAdd, // Can receive parameters step by step
  multiply,
  subtract
);

console.log(good(5)(3)); // ((5 + 3) * 2) - 1 = 15

// Or design as single-parameter functions from the start
const addN = (n) => (x) => x + n;
const multiplyN = (n) => (x) => x * n;
const subtractN = (n) => (x) => x - n;

const calc = pipe(addN(3), multiplyN(2), subtractN(1));

console.log(calc(5)); // ((5 + 3) * 2) - 1 = 15

Point-Free Style

Point-Free style means function definitions don't explicitly mention data parameters:

javascript

// ❌ Pointed style - explicitly mention parameters
const getNames = (users) => users.map((user) => user.name);

// ✅ Point-Free style - don't mention parameters
const prop = (key) => (obj) => obj[key];
const map = (fn) => (array) => array.map(fn);

const getNames = map(prop("name"));

// More complex example
const users = [
  { name: "John", age: 25, active: true },
  { name: "Jane", age: 30, active: false },
  { name: "Bob", age: 35, active: true },
];

// Pointed style
const getActiveUsersNames = (users) =>
  users
    .filter((user) => user.active)
    .map((user) => user.name)
    .map((name) => name.toUpperCase());

// Point-Free style
const filter = (predicate) => (array) => array.filter(predicate);
const toUpperCase = (str) => str.toUpperCase();

const getActiveUsersNamesPF = pipe(
  filter(prop("active")),
  map(prop("name")),
  map(toUpperCase)
);

console.log(getActiveUsersNames(users)); // ['JOHN', 'BOB']
console.log(getActiveUsersNamesPF(users)); // ['JOHN', 'BOB']

Advantages of Point-Free style:

More concise, reduces boilerplate code
Easier to compose and reuse
Focus on data transformation rather than data itself
Easier to test (no need to mock data)

Advanced Composition Patterns

1. Transducer

Transducer is an efficient method for combining multiple transformation operations, avoiding the creation of intermediate arrays:

javascript

// Traditional way - creates multiple intermediate arrays
const numbers = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10];

const traditional = numbers
  .map((x) => x * 2) // Intermediate array 1
  .filter((x) => x > 10) // Intermediate array 2
  .map((x) => x + 1); // Final array

console.log(traditional); // [11, 13, 15, 17, 19, 21]

// Transducer way - single traversal
const mapT = (fn) => (reducer) => (acc, value) => reducer(acc, fn(value));

const filterT = (predicate) => (reducer) => (acc, value) =>
  predicate(value) ? reducer(acc, value) : acc;

const transduce = (transducer, reducer, initial, collection) => {
  const xf = transducer(reducer);
  return collection.reduce(xf, initial);
};

const transducer = pipe(
  mapT((x) => x * 2),
  filterT((x) => x > 10),
  mapT((x) => x + 1)
);

const result = transduce(
  transducer,
  (acc, value) => [...acc, value],
  [],
  numbers
);

console.log(result); // [11, 13, 15, 17, 19, 21]

2. Async Composition

Handle composition of async operations:

javascript

// Async pipe
const asyncPipe =
  (...fns) =>
  (value) =>
    fns.reduce(async (acc, fn) => fn(await acc), Promise.resolve(value));

// Async operations
const fetchUser = async (id) => {
  console.log(`Fetching user ${id}...`);
  await new Promise((resolve) => setTimeout(resolve, 100));
  return { id, name: "John", email: "[email protected]" };
};

const enrichWithPosts = async (user) => {
  console.log(`Fetching posts for ${user.name}...`);
  await new Promise((resolve) => setTimeout(resolve, 100));
  return {
    ...user,
    posts: [
      { id: 1, title: "Hello World" },
      { id: 2, title: "JavaScript Tips" },
    ],
  };
};

const enrichWithComments = async (user) => {
  console.log(`Fetching comments for ${user.name}...`);
  await new Promise((resolve) => setTimeout(resolve, 100));
  return {
    ...user,
    comments: [
      { id: 1, text: "Great post!" },
      { id: 2, text: "Thanks for sharing" },
    ],
  };
};

const addTimestamp = (user) => ({
  ...user,
  fetchedAt: new Date(),
});

// Combine async operations
const getUserWithAllData = asyncPipe(
  fetchUser,
  enrichWithPosts,
  enrichWithComments,
  addTimestamp
);

getUserWithAllData(1).then((user) => {
  console.log("Complete user data:", user);
});

3. Conditional Composition

Choose different functions to compose based on conditions:

javascript

// Conditional execution
const when = (predicate, fn) => (value) => predicate(value) ? fn(value) : value;

const unless = (predicate, fn) => when((value) => !predicate(value), fn);

// Usage examples
const isNegative = (x) => x < 0;
const isLarge = (x) => x > 100;
const isDecimal = (x) => !Number.isInteger(x);

const processNumber = pipe(
  when(isNegative, Math.abs), // Convert to positive
  when(isLarge, () => 100), // Limit maximum value
  unless(Number.isInteger, Math.floor) // Round down
);

console.log(processNumber(-50)); // 50
console.log(processNumber(150)); // 100
console.log(processNumber(45.7)); // 45
console.log(processNumber(30)); // 30

Branch composition:

javascript

// Choose different processing flows based on conditions
const branch = (predicate, trueFn, falseFn) => (value) =>
  predicate(value) ? trueFn(value) : falseFn(value);

const cond =
  (...pairs) =>
  (value) => {
    for (const [predicate, fn] of pairs) {
      if (predicate(value)) {
        return fn(value);
      }
    }
    return value;
  };

// Usage examples
const categorizeAge = cond(
  [(age) => age < 13, () => "child"],
  [(age) => age < 20, () => "teenager"],
  [(age) => age < 60, () => "adult"],
  [() => true, () => "senior"]
);

console.log(categorizeAge(10)); // 'child'
console.log(categorizeAge(16)); // 'teenager'
console.log(categorizeAge(35)); // 'adult'
console.log(categorizeAge(70)); // 'senior'

4. Collector Composition

Handle aggregation operations:

javascript

// Create collectors
const collector = {
  toArray: () => (acc, value) => [...acc, value],
  toSet: () => (acc, value) => acc.add(value),
  toObject: (keyFn, valueFn) => (acc, value) => ({
    ...acc,
    [keyFn(value)]: valueFn(value),
  }),
  groupBy: (keyFn) => (acc, value) => {
    const key = keyFn(value);
    return {
      ...acc,
      [key]: [...(acc[key] || []), value],
    };
  },
};

// Usage
const users = [
  { id: 1, name: "John", role: "admin" },
  { id: 2, name: "Jane", role: "user" },
  { id: 3, name: "Bob", role: "admin" },
];

// Convert to object
const usersMap = users.reduce(
  collector.toObject(
    (u) => u.id,
    (u) => u
  ),
  {}
);

console.log(usersMap);
// { '1': {id: 1, ...}, '2': {id: 2, ...}, '3': {id: 3, ...} }

// Group by role
const groupedByRole = users.reduce(
  collector.groupBy((u) => u.role),
  {}
);

console.log(groupedByRole);
// { admin: [{id: 1, ...}, {id: 3, ...}], user: [{id: 2, ...}] }

Practical Scenarios

1. Data Validation Pipeline

javascript

// Build flexible validation system
const validate = {
  required: (field) => (value) => ({
    valid: value != null && value !== "",
    error: value ? null : `${field} is required`,
  }),

  minLength: (field, min) => (value) => ({
    valid: value && value.length >= min,
    error: value?.length >= min ? null : `${field} must be at least ${min} characters`,
  }),

  email: (field) => (value) => ({
    valid: /^[^\s@]+@[^\s@]+\.[^\s@]+$/.test(value),
    error: /^[^\s@]+@[^\s@]+\.[^\s@]+$/.test(value)
      ? null
      : `${field} format is incorrect`,
  }),

  range: (field, min, max) => (value) => ({
    valid: value >= min && value <= max,
    error:
      value >= min && value <= max
        ? null
        : `${field} must be between ${min}-${max}`,
  }),
};

// Compose validators
const composeValidators =
  (...validators) =>
  (value) => {
    const results = validators.map((v) => v(value));
    const firstError = results.find((r) => !r.valid);

    return {
      valid: !firstError,
      error: firstError?.error || null,
      errors: results.filter((r) => !r.valid).map((r) => r.error),
    };
  };

// Usage
const validateUsername = composeValidators(
  validate.required("Username"),
  validate.minLength("Username", 3)
);

const validateEmail = composeValidators(
  validate.required("Email"),
  validate.email("Email")
);

const validateAge = composeValidators(
  validate.required("Age"),
  validate.range("Age", 18, 100)
);

console.log(validateUsername("")); // { valid: false, error: '...' }
console.log(validateUsername("ab")); // { valid: false, error: '...' }
console.log(validateUsername("john")); // { valid: true, error: null }

console.log(validateEmail("invalid")); // { valid: false, error: '...' }
console.log(validateEmail("[email protected]")); // { valid: true, error: null }

2. Data Transformation Pipeline

javascript

// Build complex data processing flows
const transform = {
  filterBy: (key, value) => (items) =>
    items.filter((item) => item[key] === value),

  sortBy:
    (key, order = "asc") =>
    (items) =>
      [...items].sort((a, b) =>
        order === "asc" ? (a[key] > b[key] ? 1 : -1) : a[key] < b[key] ? 1 : -1
      ),

  mapTo: (fn) => (items) => items.map(fn),

  take: (n) => (items) => items.slice(0, n),

  unique: (key) => (items) => {
    const seen = new Set();
    return items.filter((item) => {
      const value = item[key];
      if (seen.has(value)) return false;
      seen.add(value);
      return true;
    });
  },
};

// E-commerce order processing
const orders = [
  {
    id: 1,
    customer: "John",
    amount: 150,
    status: "pending",
    date: "2025-01-01",
  },
  {
    id: 2,
    customer: "Jane",
    amount: 200,
    status: "completed",
    date: "2025-01-02",
  },
  {
    id: 3,
    customer: "Bob",
    amount: 100,
    status: "pending",
    date: "2025-01-03",
  },
  {
    id: 4,
    customer: "Alice",
    amount: 300,
    status: "completed",
    date: "2025-01-04",
  },
  {
    id: 5,
    customer: "John",
    amount: 250,
    status: "pending",
    date: "2025-01-05",
  },
];

// Get pending orders, sort by amount descending, take top 3, format output
const processOrders = pipe(
  transform.filterBy("status", "pending"),
  transform.sortBy("amount", "desc"),
  transform.take(3),
  transform.mapTo((order) => ({
    id: order.id,
    summary: `Order #${order.id} - ${order.customer}`,
    amount: `$${order.amount}`,
  }))
);

console.log(processOrders(orders));

3. React Hooks Composition

javascript

// Compose React Hooks to create custom Hook
const composeHooks = (...hooks) => (...args) => {
  return hooks.reduce((acc, hook) => {
    const result = hook(...args);
    return { ...acc, ...result };
  }, {});
};

// Example Hooks
const useLocalStorage = (key, initialValue) => {
  const [value, setValue] = useState(() => {
    const item = localStorage.getItem(key);
    return item ? JSON.parse(item) : initialValue;
  });

  const updateValue = (newValue) => {
    setValue(newValue);
    localStorage.setItem(key, JSON.stringify(newValue));
  };

  return { value, setValue: updateValue };
};

const useDebounce = (value, delay) => {
  const [debouncedValue, setDebouncedValue] = useState(value);

  useEffect(() => {
    const timer = setTimeout(() => setDebouncedValue(value), delay);
    return () => clearTimeout(timer);
  }, [value, delay]);

  return { debouncedValue };
};

// Compose usage
const useDebouncedLocalStorage = composeHooks(
  useLocalStorage,
  (key, initialValue) => useDebounce(initialValue, 500)
);

4. Middleware Pipeline

javascript

// Express-style middleware composition
const composeMiddleware = (...middlewares) => {
  return async (context) => {
    let index = 0;

    const next = async () => {
      if (index >= middlewares.length) return;

      const middleware = middlewares[index++];
      await middleware(context, next);
    };

    await next();
    return context;
  };
};

// Middleware examples
const logger = async (ctx, next) => {
  console.log(`-> ${ctx.method} ${ctx.path}`);
  await next();
  console.log(`<- ${ctx.method} ${ctx.path} ${ctx.status}`);
};

const auth = async (ctx, next) => {
  if (!ctx.user) {
    ctx.status = 401;
    ctx.body = { error: "Unauthorized" };
    return;
  }
  await next();
};

const rateLimit = async (ctx, next) => {
  // Simplified rate limiting logic
  if (Math.random() < 0.1) {
    ctx.status = 429;
    ctx.body = { error: "Too many requests" };
    return;
  }
  await next();
};

const handler = async (ctx, next) => {
  ctx.status = 200;
  ctx.body = { message: "Success", data: { id: 1 } };
  await next();
};

// Compose middleware
const app = composeMiddleware(logger, auth, rateLimit, handler);

// Usage
app({
  method: "GET",
  path: "/api/users",
  user: { id: 1, name: "John" },
}).then((result) => {
  console.log("Response:", result);
});

Performance Optimization

1. Lazy Evaluation

javascript

// Delay execution until results are needed
class LazySequence {
  constructor(data) {
    this.data = data;
    this.operations = [];
  }

  map(fn) {
    this.operations.push({ type: "map", fn });
    return this;
  }

  filter(fn) {
    this.operations.push({ type: "filter", fn });
    return this;
  }

  take(n) {
    this.operations.push({ type: "take", n });
    return this;
  }

  // Only execute when toArray is called
  toArray() {
    let result = this.data;

    for (const op of this.operations) {
      if (op.type === "map") {
        result = result.map(op.fn);
      } else if (op.type === "filter") {
        result = result.filter(op.fn);
      } else if (op.type === "take") {
        result = result.slice(0, op.n);
      }
    }

    return result;
  }
}

const lazy = (data) => new LazySequence(data);

// Usage
const numbers = Array.from({ length: 1000000 }, (_, i) => i);

const result = lazy(numbers)
  .map((x) => x * 2)
  .filter((x) => x > 100)
  .take(5)
  .toArray(); // Only starts executing now

console.log(result);

2. Function Memoization Composition

javascript

// Add memoization to composed functions
const memoizedCompose = (...fns) => {
  const cache = new Map();
  const composed = compose(...fns);

  return (value) => {
    const key = JSON.stringify(value);

    if (cache.has(key)) {
      return cache.get(key);
    }

    const result = composed(value);
    cache.set(key, result);

    return result;
  };
};

Summary

Function composition is the essence of functional programming, it allows us to:

Break down complex problems: Split big problems into small functions
Build data pipelines: pipe/compose create clear processing flows
Improve reusability: Small functions can be combined in different scenarios
Enhance testability: Each small function is tested independently
Elegant abstraction: Point-Free style and advanced patterns

Key techniques:

Basic: compose、pipe
Advanced: transducer、async composition、conditional composition
Optimization: lazy evaluation、memoization
Practice: validation、transformation、middleware

Advanced Function Composition: Building Elegant Data Processing Pipelines ​

The Art of Composition ​

Basic Composition Patterns ​

1. Compose and Pipe ​

2. Composition of Multi-parameter Functions ​

Point-Free Style ​

Advanced Composition Patterns ​

1. Transducer ​

2. Async Composition ​

3. Conditional Composition ​

4. Collector Composition ​

Practical Scenarios ​

1. Data Validation Pipeline ​

2. Data Transformation Pipeline ​

3. React Hooks Composition ​

4. Middleware Pipeline ​

Performance Optimization ​

1. Lazy Evaluation ​

2. Function Memoization Composition ​

Summary ​

Advanced Function Composition: Building Elegant Data Processing Pipelines

The Art of Composition

Basic Composition Patterns

1. Compose and Pipe

2. Composition of Multi-parameter Functions

Point-Free Style

Advanced Composition Patterns

1. Transducer

2. Async Composition

3. Conditional Composition

4. Collector Composition

Practical Scenarios

1. Data Validation Pipeline

2. Data Transformation Pipeline

3. React Hooks Composition

4. Middleware Pipeline

Performance Optimization

1. Lazy Evaluation

2. Function Memoization Composition

Summary