# Ignite Core

## Queue Configuration

Ignite programs commonly need two non-default Redis queues supervised by the
application's existing Supervisor-managed Horizon process:

- `redis-long-running` with queue `long-running` is the general long-running queue
  for existing jobs that need more time than the default queue allows.
- `redis-report-download` with queue `report-download` isolates report downloads
  from other long-running work. Report downloads can be database-heavy and
  memory-heavy, especially for XLSX files, so this queue should have its own
  worker count, timeout, retry policy, and wait alert threshold.

Programs can run report downloads on the `long-running` queue for the quickest
integration, but the recommended configuration is to keep report downloads on the
dedicated `report-download` queue.

Start by overriding the Core module queue settings in the Laravel app's
`config/core.php`. The app config should load the module defaults from
`vendor/ignite/modules/Core/config/config.php` and override only the values that
are different for the program:

```php
'report_download' => [
    'queue' => env('REPORT_DOWNLOAD_QUEUE', 'report-download'),
],
'import' => [
    'queue_connection' => env('IMPORT_QUEUE_CONNECTION', ''),
],
```

These Core config values are queue names passed to Laravel's `onQueue()` method.
They are not Laravel queue connection names. Use them with one of the queue
layouts below:

- Dedicated report downloads: set `core.report_download.queue` to
  `report-download` (or by environment variable `REPORT_DOWNLOAD_QUEUE`). Add a
  Horizon supervisor whose `connection` is `redis-report-download` and whose
  `queue` contains `report-download`.
- Long-running import queue if needed: set `core.import.queue_connection` to
  `long-running` (or by environment variable `IMPORT_QUEUE_CONNECTION`). This
  legacy config key is also a queue name. If long-running imports are not needed,
  leave it blank to use Laravel's default queue behavior.

Add the Redis queue connections required by the selected layout in the Laravel
app's `config/queue.php`:

```php
'redis-long-running' => [
    'driver' => 'redis',
    'connection' => 'default',
    'queue' => 'long-running',
    'retry_after' => 900,
    'block_for' => null,
],
'redis-report-download' => [
    'driver' => 'redis',
    'connection' => 'default',
    'queue' => 'report-download',
    'retry_after' => 3900,
    'block_for' => null,
],
```

Then configure wait thresholds and supervisors for those queues in the Laravel
app's `config/horizon.php`. The supervisor `connection` value must be the queue
connection from `config/queue.php`, and the supervisor `queue` value must contain
the queue name configured in `config/core.php`. For Laravel 7 / older Horizon,
use `delay` for retry delay:

```php
'waits' => [
    'redis:default' => 60,
    'redis-long-running:long-running' => 600,
    'redis-report-download:report-download' => 3600,
],
```

```php
'supervisor-production-2' => [
    'connection' => 'redis-long-running',
    'queue' => ['long-running'],
    'balance' => 'simple',
    'processes' => 2,
    'delay' => 60,
    'timeout' => 600,
    'tries' => 3,
    'nice' => 0,
],
'supervisor-production-3' => [
    'connection' => 'redis-report-download',
    'queue' => ['report-download'],
    'balance' => 'simple',
    'processes' => 1,
    'delay' => 60,
    'timeout' => 3600,
    'tries' => 3,
    'nice' => 0,
],
```

For Laravel 11+ / newer Horizon, use `backoff` instead:

```php
'supervisor-production-2' => [
    'connection' => 'redis-long-running',
    'queue' => ['long-running'],
    'balance' => 'simple',
    'processes' => 1,
    'backoff' => [60, 300],
    'timeout' => 600,
    'tries' => 3,
    'nice' => 0,
],
'supervisor-production-3' => [
    'connection' => 'redis-report-download',
    'queue' => ['report-download'],
    'balance' => 'simple',
    'processes' => 1,
    'backoff' => [60, 300],
    'timeout' => 3600,
    'tries' => 3,
    'nice' => 0,
],
```

- Keep each Redis `retry_after` longer than that connection's Horizon timeout so
  the same job is not delivered to two workers while it is still running.
- The Horizon `waits` values are long-wait alert thresholds for queued work, not
  job runtime timeouts; set them to the longest acceptable queue wait before
  alerts should fire.
- After changing Horizon configuration, deploy it through the program's normal
  Supervisor/Horizon restart procedure; do not start a separate queue worker
  manually.


## Report Downloads

Core can generate large CSV and XLSX files asynchronously from an existing
`QueryTable` or `EloquentTable`, or from a trusted ad-hoc Eloquent/query builder.
Every request is stored in `core_report_download`, processed by Horizon on the
configured report download queue, defaulting to `report-download`, and written to
the configured Laravel filesystem disk at `core.report_download.disk`. With the
default local disk, files are stored at:

```text
storage/app/report-downloads/{download-id}-{download-slug}.{extension}
```

The database record is the source of truth for status, measured progress,
row count, errors, cancellation, retries, file path, and expiry time. Storage
disk and row-count expectations are configuration/runtime details, so they are
not stored separately on each download record.

@todo Research the best long-term way to replace or extend the DataTables
`postCsv` button for queued report downloads without changing shared export
behavior unexpectedly.

### ReportDownloadService flow

`ReportDownloadService` accepts a `ReportDownloadRequest`. The request is a
serializable description of what should be downloaded; it never stores a live HTTP
request, query builder, session, closure, or table instance.

The service handles these request sources:

- `ReportDownloadRequest::forTable()` stores a Core `QueryTable` or `EloquentTable`
  class, constructor parameters, filters, sort, search, and visible columns. The
  service later rebuilds the table through the container, applies `FilterFactory`,
  recreates the allow-listed DataTables search/sort state, uses
  `queryForDownload()` for the filtered query, and formats rows with the table's
  `fastExcelCallback()`.
- `ReportDownloadRequest::forEloquentQuery()` stores trusted application-built
  Eloquent SQL, bindings, connection name, explicit result columns, and optional
  sort/search context. The service later rebuilds it as a subquery and writes only
  the configured result columns.
- `ReportDownloadRequest::forQueryBuilder()` does the same for Laravel query
  builder instances.

Use `expectedRowCount($request)` when callers need the exact filtered count.
Use `shouldQueue($request)` to compare that count with
`core.report_download.record_threshold`. `requestReportDownload($request)` always
creates a durable `core_report_download` record and dispatches the queued job; it
does not fall back to inline output. `process($id)` is the Horizon job path that
rehydrates the stored request, writes the CSV/XLSX file, updates progress, and
marks the record complete.

### Download an existing table

Pass the table constructor parameters, configured filters, stable sort keys, and
visible columns through `ReportDownloadRequest`. The service stores only serializable
values; it never queues a live request, builder, session, or closure.

```php
use Ignite\Core\Services\Reports\ReportDownloadRequest;
use Ignite\Core\Services\Reports\ReportDownloadService;
use Ignite\Core\Models\Report\EnrollmentReport;

$request = ReportDownloadRequest::forTable(
    auth()->id(),
    EnrollmentReport::class,
    'Enrollment Report',
    ReportDownloadRequest::FORMAT_CSV,
    (array) $report->params,
    request('filters', []),
    [['column' => 'user_id', 'direction' => 'asc']],
    ['value' => '', 'columns' => []],
    ['user_id', 'created_at']
);
```

Before queueing, use the same request object to decide whether the existing inline
download path should continue or whether the request should become a background
report download. `shouldQueue()` counts the filtered rows represented by the
request and compares that count with `core.report_download.record_threshold`.
`requestReportDownload()` always queues, so call it only after choosing the
background path.

```php
$service = app(ReportDownloadService::class);
$expectedRows = $service->expectedRowCount($request);

if (! $service->shouldQueue($request)) {
    return $this->downloadInline($request);
}

$download = $service->requestReportDownload($request);

return response()->json([
    'download_id' => $download->getKey(),
    'expected_rows' => $expectedRows,
]);
```

The same call works for CRUD tables without registration. For example:

```php
use Ignite\Core\Models\Grid\TransactionTable;

$request = ReportDownloadRequest::forTable(
    auth()->id(),
    TransactionTable::class,
    'Transactions',
    ReportDownloadRequest::FORMAT_CSV,
    [],
    request('filters', []),
    [['column' => 'transaction_date', 'direction' => 'desc']]
);
```

Constructor parameters are restored through the container. A parameterized
`EloquentTable` such as `ParticipantHistoryTable` can therefore receive its user:

```php
use Ignite\Core\Models\Grid\ParticipantHistoryTable;

$request = ReportDownloadRequest::forTable(
    auth()->id(),
    ParticipantHistoryTable::class,
    'Participant History',
    ReportDownloadRequest::FORMAT_XLSX,
    ['user_id' => $participant->user_id]
);
```

When starting from an existing `Report` record, use its stored class and params
instead of adding a report-class registry:

```php
$request = ReportDownloadRequest::forTable(
    auth()->id(),
    $report->class,
    $report->name,
    ReportDownloadRequest::FORMAT_CSV,
    (array) $report->params
);
```

To reproduce the allow-listed filter, search, sort, and visible-column state from
a DataTables HTTP request:

```php
$request = ReportDownloadRequest::fromHttpRequest(
    request(),
    auth()->id(),
    EnrollmentReport::class,
    'Enrollment Report',
    ReportDownloadRequest::FORMAT_XLSX,
    (array) $report->params
);

$download = app(ReportDownloadService::class)->requestReportDownload($request);
```

Table downloads reuse `columns()`, configured `FilterFactory` scopes, Yajra search
and sorting, and `fastExcelCallback()`. The new `queryForDownload()` seam on
`AbstractTable` applies registered scopes before the background query is built.

### Download a trusted ad-hoc query

Ad-hoc downloads require explicit result keys and headings. Build these queries only
from trusted application code. User-provided raw SQL must never be accepted.

```php
$query = \Ignite\Core\Entities\Participant::query()
    ->select(['user_id', 'first', 'last', 'email'])
    ->where('type', 'participant');

$request = ReportDownloadRequest::forEloquentQuery(
    auth()->id(),
    $query,
    'Active Participants',
    [
        'user_id' => 'User ID',
        'first' => 'First Name',
        'last' => 'Last Name',
        'email' => 'Email',
    ],
    ReportDownloadRequest::FORMAT_CSV,
    [['column' => 'last', 'direction' => 'asc']]
);

$download = app(ReportDownloadService::class)->requestReportDownload($request);
```

Use `ReportDownloadRequest::forQueryBuilder()` in the same way for a Laravel query
builder. The builder is reduced to its SQL, bindings, connection name, and explicit
columns so the job payload remains small and queue-safe.

### DataTables CSV button integration

Core includes `resources/assets/js/report-download-buttons.js` for pages that want
the existing `postCsv` button to queue large exports through
`ReportDownloadService`. The script only changes pages that load it.

The button posts the current DataTables search, column search, and sorting state to
the table's sibling `/export` route. It also sends `Accept: application/json` and
`X-Requested-With: XMLHttpRequest` so Laravel 7 treats the request like the normal
DataTables ajax request. If the filtered row count is below
`core.report_download.record_threshold`, the controller falls through to the
existing inline CSV response. If the count meets the threshold, Core queues a
`core_report_download` record and returns a JSON flash message.

Controllers can share the same behavior with `QueuesReportDownloads`:

```php
$queuedCsvResponse = $this->queuedDataTableCsvExportResponse(
    $request,
    'admin.participant.index.export',
    ParticipantTable::class,
    'Participants',
    ['status' => $statusLookup[$status]]
);

if ($queuedCsvResponse) {
    return $queuedCsvResponse;
}
```

Call this before the normal `$request->wantsJson()` table response. A `null`
return value means the controller should keep using the existing inline export path.
