Wait for a BullMQ job to complete with Remix Deferred Loaders

This article was written for Remix 2, which has been merged into React Router 7. Some changes may be necessary.

Remix's Defer API is a feature that allows you to return an unresolved Promise from a loader. The page will server-side render without waiting for the promise to resolve, and then when it finally does, the client will re-render with the new data.

This works especially well with queued jobs, where a user may try to look at result data before the job has finished processing. In this case, we can return a Promise that will resolve when the job is finished, and the client will re-render with the new data.

BullMQ is a modern, fast, and robust queue system for Node.js. It is a successor to the popular Bull library, and is built on top of Redis. If you've integrated BullMQ into your app, you can use the Defer API to return a Promise that will resolve when the job is finished.

import { defer, redirect } from "@remix-run/node"
import { processItemQueue } from "~/queue.server.ts"
 
export async function loader({
  params,
}: LoaderFunctionArgs) {
  const hash = params.hash
  if (!hash) {
    return redirect("/")
  }
 
  const job = await processItemQueue.getJob(hash)
  if (!job) {
    return redirect("/")
  }
 
  const TIMEOUT_MILLISECONDS = 30 * 1000
 
  return defer({
    job: job.waitUntilFinished(
      processItemQueue.events,
      TIMEOUT_MILLISECONDS,
    ),
  })
}
 
export default function Index() {
  const data = useLoaderData()
 
  return (
    <div>
      <Suspense fallback={<p> loading… </p>}>
        <Await
          resolve={data.job}
          errorElement={<p>Error loading job</p>}
        >
          {(results) => <pre>{results}</pre>}
        </Await>
      </Suspense>
    </div>
  )
}

As a next step, you can use event streams to update the page as the job progresses.