TSV EXPORTER

Introduction

Available only on MibFront, it enables the export of data in file format from a list page. It is possible to export data in two ways: one for the current page and another to export all pages.

Distributions

Tsv Exporter Agent
- Technology: Utilizes MibAgentWebHost
- Included export modules: Front, Edit History
- Go to documentation
Tsv Exporter Worker (Beta)
- Technology: Utilizes .NET WorkerService
- Included export modules: Front
- Go to documentation

Click on the documentation link for the chosen distribution to get started.

Architecture

MibServer3 and the TSV export service are decoupled processes. There is no direct call, HTTP endpoint or message broker between them. They integrate through two shared resources only:

a shared queue table in the front database (the same database MibServer3 uses) — MIB3UX_TSV_REQUESTS for the Front module and MIB3UX_TSV_EDITHISTORY_REQUESTS for the Edit History module;
a shared file store where the finished .TSV file is written (disk, SMB, S3, or the File Management microservice).

MibServer3 only writes a job row; the export service polls for it. The export service never reads CMS content from the database directly — it re-fetches the data over HTTP from MibServerApi, authenticating against MibAuthorization and applying the requester's permissions that were captured on the job row. This is why MibServerApi and MibAuthorization are required for exports to run.

flowchart LR
    subgraph CMS["MibServer3 (CMS UI + API)"]
        WF["Export request<br/>TsvExportWorkflow"]
        DL["Download<br/>FileV2Controller"]
    end

    QUEUE[("Front DB queue<br/>MIB3UX_TSV_REQUESTS<br/>MIB3UX_TSV_EDITHISTORY_REQUESTS")]
    STORE[("File store<br/>disk · SMB · S3 ·<br/>FileManagement MS")]

    subgraph EXP["TSV Exporter — Worker or Agent"]
        Q["Jobs queue manager<br/>claim · abandon · retry"]
        E["Export module<br/>fetch → format → write"]
    end

    AUTH["MibAuthorization"]
    API["MibServerApi"]

    WF -->|"INSERT job (Newjob)"| QUEUE
    QUEUE -->|"poll / claim on heartbeat"| Q --> E
    E -->|"OAuth"| AUTH
    E -->|"paged ListData<br/>criteria + permissions"| API
    E -->|"write .TSV"| STORE
    E -->|"Completed / Error + notification"| QUEUE
    QUEUE -.->|"notification"| DL
    DL -->|"stream file"| STORE

Components

Side	Component	Responsibility
MibServer3	`TsvExportController`, `ListController.ExportRequest`, `EditHistoryController`, `PageHistoryV2Controller`	Accept the export request from the UI
MibServer3	`TsvExportWorkflow`	Resolve permissions, owner timezone and API URL; build the job
MibServer3	`TsvExportRepository`	De-duplicate and insert the job row into the queue table
Shared	`MIB3UX_TSV_REQUESTS` / `MIB3UX_TSV_EDITHISTORY_REQUESTS`	Job queue plus status and agent-ownership record
Shared	`TsvExportInformation`	The job contract compiled by both sides
Exporter	Jobs queue manager / repository	Claim new, resume pending, reassign abandoned jobs
Exporter	Front / Edit History export module	Fetch from MibServerApi, format TSV, write the file
MibServer3	`FileV2Controller` (`/api/v2/file/tsvdownload/{notificationId}`)	Stream the finished file to the user

End-to-end flow

sequenceDiagram
    participant U as User (MibFront)
    participant S as MibServer3
    participant DB as Front DB queue
    participant X as TSV Exporter
    participant Au as MibAuthorization
    participant Api as MibServerApi
    participant FS as File store

    U->>S: Request export (current page / all pages)
    S->>S: Resolve permissions, timezone, API URL
    S->>DB: INSERT job — Newjob, no agent
    Note over S,DB: Duplicate (same owner + criteria) is rejected
    S-->>U: Request queued

    loop every heartbeat
        X->>DB: pending(mine) / abandoned / new ?
    end
    X->>DB: Claim job — set agent, status Pending
    X->>Au: Authenticate (OAuth)
    loop each page (batches of 250 for "all pages")
        X->>Api: ListData(criteria + permission set)
        Api-->>X: Page of items
        X->>FS: Append formatted TSV to temp file
        X->>DB: Refresh last activity (keep-alive)
    end
    X->>FS: Save final .TSV
    X->>DB: Completed (or Error) + notification

    U->>S: Open notification → download
    S->>FS: Read file
    S-->>U: Stream .TSV

Request — From a list page (or the Edit History dashboard) the user exports the current page or all pages. A MibServer3 controller passes the search criteria to TsvExportWorkflow.
Enqueue — The workflow snapshots the requester's permission set, owner timezone and data API URL onto the job and inserts it with status Newjob and no owning agent. A duplicate request (same owner + criteria still Newjob/Pending) is rejected.
Claim — Every export instance polls on a configurable heartbeat. The queue manager picks, in order: a job still pending for this instance, then an abandoned job (no progress within the configured window — reassigned, retry count incremented), then a new job. Claiming assigns the agent and sets status Pending.
Fetch — The module authenticates against MibAuthorization, then calls MibServerApi page by page (all-pages exports are fetched in batches of 250) using the stored criteria and permission set, so content authorization reflects the requester at request time.
Write — Each page is formatted to TSV (configurable separator, date/time formats, owner timezone) and appended to a temporary file. The job's last-activity timestamp is refreshed every page so a long-running export is not treated as abandoned.
Finish — The final file is persisted to the configured store; the job is marked Completed, or Error with a message on failure. Either way a notification is created for the owner.
Download — The user opens the notification in MibServer3, which streams the file from the store via FileV2Controller (/api/v2/file/tsvdownload/{notificationId}).

Note

The same Front export logic ships in two interchangeable distributions — the Worker (standalone .NET WorkerService, multi-instance) and the Agent (in-process plugin of MibAgentWebHost). Both consume the same MIB3UX_TSV_REQUESTS queue and can be scaled horizontally. The Edit History module is available only in the Agent distribution and uses the separate MIB3UX_TSV_EDITHISTORY_REQUESTS queue.

Permissions & authorization

The export service separates identity from data scope — they are handled by two independent mechanisms.

Authentication (transport identity). The Worker/Agent authenticates to MibAuthorization with its own configured service account (granttype/clientid/clientsecret/username/password). The resulting token identifies the export service, not the user who requested the export.
Authorization (data scope). The rows an export may contain are scoped by the original requester's permissions, captured as a snapshot at request time and replayed independently of the service token.

The permission-set / `X-PermissionKey` mechanism

This is what lets a background service, authenticated as a service account, return exactly the data the requesting user was allowed to see:

Capture (request time, MibServer3). When the export is requested, TsvExportWorkflow calls the API client's GetPermissions and stores the returned PermissionSetContainer (the requester's effective media-type and source permissions) on the job row (MIB3UX_TSV_REQUESTS.PERMISSION_SET).
Replay (export time, exporter → MibServerApi). When fetching list data (ItemRepository.GetList(criteria, permissionSet)), the exporter calls MibServerApi CreatePermissionSet (a POST temporarydata), which stores the serialized set under a generated key. That key is sent on the data request as the X-PermissionKey header and deleted afterwards (RemoveTemporaryData).
Override (MibServerApi). MibPermissionRulesMiddleware detects X-PermissionKey and runs PermissionOverride: it loads the stored set and replaces the caller's MediaTypePermissions/SourcePermissions for that request. All row/source SQL filtering then applies the requester's scope — even though the bearer token is the service account's.

sequenceDiagram
    participant S as MibServer3<br/>(request time)
    participant DB as Front DB queue
    participant X as TSV Exporter
    participant Api as MibServerApi
    S->>S: GetPermissions() → PermissionSetContainer (requester scope)
    S->>DB: persist PERMISSION_SET on the job row
    X->>DB: claim job (reads the persisted permission set)
    X->>Api: CreatePermissionSet(set) → POST temporarydata → key
    X->>Api: list request — header X-PermissionKey: key
    Api->>Api: PermissionOverride — replace caller perms with the set
    Api-->>X: rows scoped to the requester
    X->>Api: RemoveTemporaryData(key)

Safety properties (enforced by PermissionOverride.ValidateRequirements):

Subset only. The supplied set must fall within the authenticating principal's own permissions; it can only narrow scope, never escalate. A set exceeding the caller's permissions is rejected (PermissionSetDoesNotFulfillRequirementsException / PermissionSetMissingDataException).
No admin. IsAdministrator is forced false whenever an override is applied.
Request-time snapshot. The permission set is captured when the export is requested. If the requester's permissions change before a long-queued or retried job runs, the export still uses the original snapshot. The stored key has no TTL — the exporter creates and deletes it per fetch.

Which services are involved

Note

The X-PermissionKey / permission-set override is owned entirely by MibServerApi. It is not implemented by MibAuthorization or the MibPermission microservice, and the React front end is not involved.

Service	Role in this mechanism
MibServerApi	Owns it. Server side: `MibPermissionRulesMiddleware` + `PermissionOverride` + the `API_TEMPORARY_DATA_SET` store. Client SDK (`MediaiBox.Cms.Api.Client`): `CreatePermissionSet` / `RemoveTemporaryData` and the `X-PermissionKey` header.
MibServer3 / FrontEnd	Consumer only. Captures the `PermissionSetContainer` (`TsvExportWorkflow`) and replays it through the API client (`ItemRepository`); does not implement the override.
MibAuthorization	Issues the service-account token (transport identity) and is a source of a user's effective permissions, but has no knowledge of permission sets or `X-PermissionKey`.
MibPermission microservice	Computes/compiles a user's effective permissions (the data that can be packaged into a set). Not part of the override path.

Note

Because the override is applied by MibServerApi, this mechanism scopes only data fetched through MibServerApi. Rows a list page obtains from a different backend are not scoped by X-PermissionKey and must enforce authorization by other means.

Configuration

MibTsvExportAgentConfig

granttype, string: Type of authentication, which can be password, client_credentials, or authorization_code, commonly used password for tsvExport.
clientid, string: Application client ID.
clientsecret, string: Application client secret.
username, string: Authentication username.
password, string: Authentication password.
authorizationurl, string: Url access for authorization server.
apiurl, string: Url access for api server.
temporarytsvfilefolder, string: Path where temporary TSV files are stored.
tsvsubfolder, string: Path of the subfolder where TSV files are located, by default, it is empty.
tsvexporturl, string: URL enabled to download the file where it is stored.
storageType, string: Localization type, where the file is stored, can be disk, amazon, or SMB. The default is disk.
maximumExportWaitMinutes, int: The maximum time limit allowed for attempting a file export, the default, is 120 minutes.
retryAttempts, int: Variable to determine how many times it should retry the call before considering it a permanent failure, the default is 5 attempts.
retryAwaitSeconds, int: Representing the number of seconds to wait before making a new attempt, default is 10 seconds.
mibFrontUrl, string: Url acess for Mib3 (to get the dictionaries).
useFileManagementService: Indicates whether the FileManagement microservice should be used to upload files. (default = false) only from version MIB 6.0
authTokenEarlyRenewalMinutes: Sets secure time margin before authorization token has expired so that it can be renewed sonner rather than later. (default = 10)
mibApiCallTimeoutSeconds: defines timeout of batch processings from MibApi. (default = HttpClient default timeout)
mibAuthCallTimeoutSeconds: defines timeout when calling MibAuthorization. (default = HttpClient default timeout)
mibFileManagementCallTimeoutSeconds: defines timeout when calling MibFileManagement. (default = HttpClient default timeout)
minutesOfAgentInactivityToConsiderJobAbandoned: defines for how long an exportation job should not be progressing so that it is considered abandoned and reassigned to another agent (default = 5 minutes, minimum = 1 [not recommended]).
dateFormat (default = yyyy/MM/dd) only from version MIB 6.0
dateTimeFormat (default = yyyy/MM/dd HH:mm:ss) only from version MIB 6.0
separatorCharacter - defines which character will be used as a separator when writing files (default = \t) only from version MIB 6.0
generateDetailedLogFormattingColumns - defines whether to generate logs detailing the process of formatting the information coming from MibApi (default = false) only from version MIB 6.0
fileUploadBufferSizeBytes - defines buffer size in bytes for uploading files via FileManagement service (default = 1048576)

The "dateFormat" and "dateTimeFormat" are used to format the AdmFields types: Date, DateTime and UtcDateTime.

MibEditHistoryTsvExportertAgentConfig

granttype, string: Type of authentication, which can be password, client_credentials, or authorization_code, commonly used password for tsvExport.
clientid, string: Application client ID.
clientsecret, string: Application client secret.
username, string: Authentication username.
password, string: Authentication password.
authorizationurl, string: Url access for authorization server.
apiurl, string: Url access for api server.
temporarytsvfilefolder, string: Path where temporary TSV files are stored.
tsvsubfolder, string: Path of the subfolder where TSV files are located, by default, it is empty.
tsvexporturl, string: URL enabled to download the file where it is stored.
storageType, string: Localization type, where the file is stored, can be disk, amazon, or SMB. The default is disk.
mibFrontRootUrl, string: Url acess for Mib3 (to get the dictionaries).

MibFileManagementMicroServiceClientConfig

only from version MIB 6.0

For more details check click here

⚠️ Attention ⚠️

When using the File Management Service in TSV, the upload process used is the simplified upload. Learn more about how it works in this documentation

Export Modules

A TSV export service can support multiple export modules. Each module handles different types of export requests, which may require query data in various formats and from different sources in order to generate the export file.

Front Module

This module process export requests that comes from all places other than the Edit History. It queries the table MIB3UX_TSV_REQUESTS for new jobs. See the documentation of the TSV exporter distribution you are using for instructions on how to enable this module.

Export request format

Below is the export request format accepted by this module:

Current page

criteria: 
{
  "Page": 0,
  "Limit": "20",
  "MediaType": "movie_contents",
  "Clause": "",
  "Order": "ID,desc",
  "Conditions": [],
  "ImplicitConditions": [],
  "DisplayFields": [
    "ID",
    "NAME",
    "INSTANCE_ID",
    "OWNER",
    "MOVIE_TYPE_ID",
    "MOVIE_VERSION_ID",
    "CONTENT_TYPE_ID",
    "VOD_OFFER_TYPE_ID",
    "SOURCE_ID"
  ],
  "IsAdvancedSearch": false,
  "SearchFavoriteId": 0,
  "SourcePermission": {
    "Write": "none",
    "Delete": "none"
  },
  "ReturnUrl": "https://localhost/MibFront/Display/movie_content_list",
  "ContentCriteriaTemplateComponentKey": "",
  "SelectedSources": [],
  "DisplayValuePattern": "",
  "TemplateComponentKey": "one_list_template_list_1",
  "ShowOnlySelectedItems": false
}

All pages To export all pages, the “Limit” field is set to -1. When this option is selected, items are exported in batches of 250 until all items have been fetched from MibApi.

Edit History Module

This module process export requests related to edition history. It queries the table MIB3UX_TSV_EDITHISTORY_REQUESTS for new jobs. See the documentation of the TSV exporter distribution you are using for instructions on how to enable this module.

Export job status

The STATUS column of both queue tables (MIB3UX_TSV_REQUESTS and MIB3UX_TSV_EDITHISTORY_REQUESTS) uses the same values:

0 (NULL) → Newjob
1 → Completed
2 → Pending
3 → Error

Table of Contents

TSV EXPORTER

Introduction

Distributions

Architecture

Components

End-to-end flow

Note

Permissions & authorization

The permission-set / X-PermissionKey mechanism

Which services are involved

Note

Note

Configuration

MibTsvExportAgentConfig

MibEditHistoryTsvExportertAgentConfig

MibFileManagementMicroServiceClientConfig

Export Modules

Front Module

Export request format

Edit History Module

Export job status

The permission-set / `X-PermissionKey` mechanism