# Backend Uploading

Canonical: https://docs.projectbabbage.com/docs/quickstarts/metamarket/backend-uploading

Admit PushDrop tokens and store file metadata via your overlay’s Topic Manager and Lookup Service.

This page shows the backend changes needed to admit and store the transaction outputs created when a user uploads a file. You'll:

1. Rename your Topic Manager and Lookup Service
2. Define the UTXO payload fields
3. Configure the Topic Manager to admit those fields
4. Extend storage to persist all file metadata
5. Parse and store admitted outputs in your Lookup Service
6. Add documentation for both services
7. Verify everything is wired and linked properly 

---

## 1. Rename and register services

First, install the `@bsv/sdk` in your backend directory.

```ts
cd backend
npm install @bsv/sdk
```

After that, rename your files under `backend/src/topic-managers/` and `backend/src/lookup-services/`:

```text
backend/src/
├── topic-managers/
│   ├── MarketTopicDocs.md.ts
│   └── MarketTopicManager.ts
└── lookup-services/
    ├── MarketLookupDocs.md.ts
    ├── MarketStorage.ts
    └── MarketLookupServiceFactory.ts
```

Then update **deployment-info.json** to use unique names:

```diff
 {
   "topicManagers": {
-    "tm_template": "./backend/src/topic-managers/TemplateTopicManager.ts"
+    "tm_demo_market": "./backend/src/topic-managers/MarketTopicManager.ts"
   },
   "lookupServices": {
-    "ls_template": {
+    "ls_demo_market": {
       "serviceFactory": "./backend/src/lookup-services/MarketLookupServiceFactory.ts",
       "hydrateWith": "mongo"
     }
   },
   …
 }
```

> Make sure that the name you give it is fully unique to your project! LARS and CARS will not work if service names overlap

---

## 2. Locking script payload fields

Your frontend will embed these fields into the PushDrop payload. When decoded by your Topic Manager, expect:

-   `fileUrl` – UHRP hash of the encrypted STL file.
-   `name` – file/listing name.
-   `description` – store page description.
-   `satoshis` – sale price.
-   `publicKey` – uploader's identity key.
-   `fileSize` – size of STL in bytes.
-   `expiryTime` – Unix-ms timestamp when file expires.
-   `coverUrl` – UHRP hash of the (unencrypted) cover image.

---

## 3. Configure the Topic Manager

In **MarketTopicManager.ts**, update `identifyAdmissibleOutputs` to extract and admit all fields:

```ts
// backend/src/topic-managers/MarketTopicManager.ts
import { AdmittanceInstructions, TopicManager } from '@bsv/overlay'
import { Transaction, PublicKey, PushDrop, Utils } from '@bsv/sdk'
import docs from './MarketTopicDocs.md.js'

export default class MarketTopicManager implements TopicManager {
  async identifyAdmissibleOutputs(beef: number[], previousCoins: number[]): Promise<AdmittanceInstructions> {
  const admissibleOutputs: number[] = [];

  try {
    const decodedTx = Transaction.fromBEEF(beef)
    const outputs = decodedTx.outputs

    // Try to decode and validate transaction outputs
    for (const [index, output] of outputs.entries()) {
      try {
        const decodedScript = PushDrop.decode(output.lockingScript)
        const fields = decodedScript.fields
        
        // checking if all of the fields are structured as expected
        // UHRP Url
        if (fields[0].length !== 52) {
          console.log('Invalid UHRP url length.')
          continue
        }

        // Name
        if (fields[1].length === 0) {
          console.log('Name field is empty:', fields[1].toString())
          continue
        }

        // Description
        // (Optional validation can be added here if needed)

        // Satoshis
        const satoshis = parseInt(fields[3].toString(), 10)
        if (isNaN(satoshis) || satoshis <= 0) {
          console.log('Invalid satoshis value.')
          continue
        }

        // Public Key
        if (!fields[4] || fields[4].length === 0) {
          console.log('Public key is missing.')
          continue
        }
        try {
          console.log('Public Key:', Utils.toUTF8(Utils.toArray(fields[4])))
          PublicKey.fromString(Utils.toUTF8(Utils.toArray(fields[4])))
        } catch {
          console.log('Invalid public key format.')
          continue
        }

        // File Size
        const fileSize = parseInt(fields[5].toString(), 10)
        if (isNaN(fileSize) || fileSize <= 0) {
          console.log('Invalid file size.')
          continue
        }

        // Expiry Time
        const expiryTime = parseInt(Utils.toUTF8(Utils.toArray(fields[6])), 10)
        if (isNaN(expiryTime) || expiryTime <= Date.now()) {
          console.log('Invalid expiry time:', expiryTime)
          continue
        }

        // Cover Image UHRP Url
        if (fields[7].length !== 52) {
          console.log('Invalid cover image UHRP url length.')
          continue
        }

        admissibleOutputs.push(index)
      } catch (error) {
        continue
      }
    }
    if (admissibleOutputs.length === 0) {
      console.warn('No outputs admitted!')
    }
  } catch (error) {
    console.error('Error identifying admissible outputs:', error)
  }

  return {
    outputsToAdmit: admissibleOutputs,
    coinsToRetain: previousCoins
  }
}
```

**What's happening:**

1.  Decode PushDrop payload
2.  Convert fields into typed values
3.  Ensure the values of each of the eight fields are properly formatted
4.  Admit the output index of the valid token

---

## 4. Extend Lookup Storage

The first thing that you will want to do is to add a new interface for the records that are being stored. This interface will go along with the existing `UTXOReference` interface in **types.ts**:

```ts
// backend/src/types.ts
export interface MarketRecord {
  fileUrl: string
  name: string
  description: string
  satoshis: number
  creatorPublicKey: string
  size: number
  txid: string
  outputIndex: number
  retentionPeriod: number
  coverUrl: string
  createdAt: Date // optional field
}

export interface UTXOReference {
  txid: string
  outputIndex: number
}
```

In **TemplateStorage → MarketStorage.ts**, update `storeRecord` signature and add the query `isFileUrlInDatabase`:

```ts
// backend/src/lookup-services/MarketStorage.ts
import { Collection, Db } from 'mongodb'
import { MarketRecord, UTXOReference } from '../types'

export class MarketStorage {
  private readonly records: Collection<MarketRecord>

  constructor(private readonly db: Db) {
    this.records = db.collection<MarketRecord>('market_record')
  }

  async storeRecord(
    fileUrl: string, 
    name: string, 
    description: string, 
    satoshis: number, 
    creatorPublicKey: string, 
    size: number, 
    txid: string, 
    outputIndex: number, 
    retentionPeriod: number, 
    coverUrl: string
  ): Promise<void> {
    // logging when the values are being stored in the 
    console.log("Storing record in MongoDB:", {
      fileUrl,
      name,
      description,
      satoshis,
      creatorPublicKey,
      size,
      txid,
      outputIndex,
      retentionPeriod,
      coverUrl,
      createdAt: Date.now()
    })
    try {
      // Insert new record
      await this.records.insertOne({
        fileUrl,
        name,
        description,
        satoshis,
        creatorPublicKey,
        size,
        txid,
        outputIndex,
        retentionPeriod,
        coverUrl,
        createdAt: new Date()
      })
    } catch (error) {
      console.error("Failed to store record:", error)
      throw error
    }
  }

  // … other existing functions (deleteRecord, findAll, etc...) …

  // custom query function
  async isFileUrlInDatabase(fileUrl: string): Promise<UTXOReference[]> {
    return await this.records
      .find({ fileUrl })
      .project<UTXOReference>({ txid: 1, outputIndex: 1 })
      .toArray()
      .then(results => results.map(r => ({
        txid: r.txid, outputIndex: r.outputIndex
      })))
  }
}
```

The `isFileUrlInDatabase` query will be used to check if any of the outputs that are stored in the market's database have the provided fileUrl as a field. This will be used on file upload by the key-server to verify the admittance. Note that it does not return a boolean because **all** queries in the storage class should have the return type `UTXOReference`.

---

## 5. Implement Lookup Service Factory

In **MarketLookupServiceFactory.ts**, parse all fields and call `storeRecord`:

```ts
// backend/src/lookup-services/MarketLookupServiceFactory.ts
import {
  LookupService, OutputAdmittedByTopic, OutputSpent,
  LookupQuestion, LookupAnswer, LookupFormula,
  AdmissionMode, SpendNotificationMode
} from '@bsv/overlay'
import { PushDrop, Utils } from '@bsv/sdk'
import { MarketStorage } from './MarketStorage.js'
import docs from './MarketLookupDocs.md.js'
import { Db } from 'mongodb'


/**
 * Template for a Lookup Service that can be modified for your specific use-case.
 */
export class MarketLookupService implements LookupService {
  readonly admissionMode: AdmissionMode = 'locking-script'
  readonly spendNotificationMode: SpendNotificationMode = 'none'

  constructor(public storage: MarketStorage) { }

  async outputAdmittedByTopic(payload: OutputAdmittedByTopic): Promise<void> {
    if (payload.mode !== 'locking-script') throw new Error('Invalid mode')
    const { topic, lockingScript, txid, outputIndex } = payload
    if (topic !== 'tm_demo_market') throw new Error(`Invalid topic "${topic}" for this service.`)
    try {
      const decodedScript = PushDrop.decode(lockingScript)
      const fields = decodedScript.fields

      const uhrpUrl = Utils.toUTF8(Utils.toArray(fields[0]))
      const name = Utils.toUTF8(Utils.toArray(fields[1]))
      const description = Utils.toUTF8(Utils.toArray(fields[2]))
      const satoshis = Number(Utils.toUTF8(Utils.toArray(fields[3])))
      const creatorPublicKey = Utils.toUTF8(Utils.toArray(fields[4]))
      const size = Number(Utils.toUTF8(Utils.toArray(fields[5])))
      const retentionPeriod = Number(Utils.toUTF8(Utils.toArray(fields[6])))
      const coverUrl = Utils.toUTF8(Utils.toArray(fields[7]))

      // Store the token fields for future lookup
      await this.storage.storeRecord(
        uhrpUrl,
        name,
        description,
        satoshis,
        creatorPublicKey,
        size,
        txid,
        outputIndex,
        retentionPeriod,
        coverUrl
      )
    } catch (error) {
      console.error('Failed to store record', error)
      return
    }
  }
  // … other existing methods …
}

// factory
export default (db: Db) =>
  new MarketLookupService(new MarketStorage(db))
```

**Key steps:**

1.  Decode PushDrop fields array
2.  Map to typed variables
3.  Call `storeRecord` with all metadata

### Adding the query

In **MarketLookupServiceFactory.ts**, add this helper function after the class definition to check the type of the query:

```
// helper functions

function isUrlCheckQuery(query: any): query is { type: 'urlCheck'; value: { fileUrl: string } } {
  return (
    typeof query === 'object' &&
    query.type === 'urlCheck' &&
    query.value &&
    typeof query.value.fileUrl === 'string'
  );
}
```

And add the query in the `lookup` function:

```ts
async lookup(question: LookupQuestion): Promise<LookupAnswer | LookupFormula> {
  const query = question.query
  
  // Validate query presence
  if (!query) {
    throw new Error('A valid query must be provided!');
  }
  
  // Validate service
  if (question.service !== 'ls_demo_market') {
    throw new Error('Lookup service not supported!');
  }

  // Handle specific queries
  if (query === 'findAll') {
    return await this.storage.findAll();
  }  

  // Handles the query for a specific fileUrl
  if (isUrlCheckQuery(query)) {    
    const { fileUrl } = query.value
    return await this.storage.isFileUrlInDatabase(fileUrl)
  }
  
  throw new Error('Unknown query type');
}
```

---

## 6. Adding Documentation

Proper documentation can help users understand what your Overlay Services and will be shown in the UI for the backend server.

### 6.1 Topic Manager Docs

In `backend/src/topic-managers/MarketTopicManager.ts`, replace the placeholder shortDescription:

```ts
async getMetaData(): Promise<{
  name: string
  shortDescription: string
  iconURL?: string
  version?: string
  informationURL?: string
}> {
  return {
    name: 'Marketplace Topic Manager',
    shortDescription: 'Admits and filters tokens for the 3D marketplace'
  }
}
```

And add to `MarketTopicDocs.md.ts` with a description of this Topic Manager.

### 6.2 Lookup Service Docs

In `backend/src/lookup-services/MarketLookupServiceFactory.ts`:

```
async getMetaData(): Promise<{
  name: string
  shortDescription: string
  iconURL?: string
  version?: string
  informationURL?: string
}> {
  return {
    name: 'Marketplace Lookup Service',
    shortDescription: 'Provides search and retrieval of admitted 3D-object listings'
  }
}
```

And add to `MarketLookupDocs.md.ts` with a description of this Lookup Service.

---

## 7. Verify Your Overlay Services

Now launch LARS and confirm your new Topic Manager and Lookup Service are loaded:

```bash
# from your project root
npx lars
```

Open your browser to [http://localhost:8080](http://localhost:8080)
In the **Topic Managers** and **Lookup Services** menus you should see entries for:

- **tm\_demo\_market** → MarketTopicManager
- **ls\_demo\_market** → MarketLookupService

If they appear there, your backend wiring is complete! Next up: configuring the key server with middlewares and routes for uploading.