# Key Server Uploading

Canonical: https://docs.projectbabbage.com/docs/quickstarts/metamarket/key-server-uploading

Verify on-chain commitments and persist encryption keys and payment data in your key server.

This page walks through setting up your Key Server to handle file‐upload submissions for the Metanet 3D Marketplace: installing dependencies, wiring environment variables, configuring wallet access, middleware, decoding on-chain payloads, and persisting metadata.

---

## 1. Install dependencies & configure `.env`

In your `key-server/` directory, install required packages:

```bash
cd key-server
npm install express body-parser dotenv @bsv/sdk @bsv/wallet-toolbox \
  @bsv/auth-express-middleware @bsv/payment-express-middleware mongodb prettyjson
npm install --save-dev typescript ts-node-dev @types/node @types/express @types/body-parser
```

Update the `type` and your `start` script in  `key-server/package.json`:

```json
"scripts": {
- "start": "ts-node-dev --respawn src/index.ts"
+ "start": "node --loader ts-node/esm/transpile-only --max-http-header-size=10485760 src/index.ts"
},
```

> This replaces the ts-node-dev script added earlier.

```json
{
- "type": "commonjs",
+ "type": "module",
}
```


Create a `.env` file in `key-server/` with:

```dotenv
PORT=3000
SERVER_PRIVATE_KEY=your-server-private-key
SERVER_URL=http://localhost:3000/
WALLET_STORAGE_URL=https://storage.babbage.systems

## MongoDB Configuration
MONGO_URI=mongodb://localhost:27017
DATABASE_NAME=your-database-name
KEY_COLLECTION_NAME=files
BALANCE_COLLECTION_NAME=balance

BSV_NETWORK=mainnet # or testnet
NETWORK_PRESET=local # or mainnet/testnet for deployment with CARS
```

> generate a unique SHA-256 hash as your private key and rename the database


---

## 2. Define Key Server types

Add `key-server/src/types.ts`:

```ts
// key-server/src/types.ts
export interface DecodedOutput {
  fileUrl?: string
  name?: string
  description?: string
  satoshis?: number
  creatorPublicKey?: string
  size?: number
  txid: string
  outputIndex: number
  retentionPeriod?: number
  coverUrl?: string
}

export interface KeyRecord {
  fileUrl: string
  encryptionKey: string
  satoshis: number
  publicKey: string
}
```

---

## 3. Create the getWallet util

We'll start by creating some helper utility functions and classes that we will later utilize in `index.ts`

Provide your server with a BSV wallet using `@bsv/wallet-toolbox`, this wallet will need to be funded at [WUI](https://wui.bapp.dev/) using the private key you generated in the `.env` file:

```ts
// key-server/src/utils/walletSingleton.ts
import { KeyDeriver, PrivateKey, WalletInterface } from '@bsv/sdk'
import { Services, StorageClient, Wallet, WalletSigner, WalletStorageManager } from '@bsv/wallet-toolbox';

const walletInstance: WalletInterface | null = null

export async function getWallet(): Promise<WalletInterface> {
  const SERVER_PRIVATE_KEY = process.env.SERVER_PRIVATE_KEY as string
  const WALLET_STORAGE_URL = process.env.WALLET_STORAGE_URL as string
  const BSV_NETWORK = process.env.BSV_NETWORK as 'mainnet' | 'testnet'

  if (!walletInstance) {
    const chain = BSV_NETWORK === 'mainnet' ? 'main' : 'test'
    const keyDeriver = new KeyDeriver(new PrivateKey(SERVER_PRIVATE_KEY, 'hex'));
    const storageManager = new WalletStorageManager(keyDeriver.identityKey);
    const signer = new WalletSigner(chain, keyDeriver, storageManager);
    const services = new Services(chain);
    const wallet = new Wallet(signer, services);
    const client = new StorageClient(wallet, WALLET_STORAGE_URL);
    await client.makeAvailable();
    await storageManager.addWalletStorageProvider(client);
    return wallet;
  }
  return walletInstance
}
```

### What's happening:

- Deriving the server identity key and setting it as the server's wallet's key
- Wire up on-chain signer and storage client
- Cache a single wallet instance that will be used throughout the project

---

## 4. Implement KeyStorage

Create `key-server/src/KeyStorage.ts` to persist file records:

```ts
// key-server/src/KeyStorage.ts
import { Collection, Db, DeleteResult } from 'mongodb';
import { KeyRecord } from './types.js';
import dotenv from 'dotenv'

dotenv.config()

const KEY_COLLECTION_NAME = process.env.KEY_COLLECTION_NAME as string

export class KeyStorage {
  private readonly records: Collection<KeyRecord>

  constructor(private readonly db: Db) {
    this.records = db.collection<KeyRecord>(KEY_COLLECTION_NAME)
  }

  async storeRecord(
    fileUrl: string,
    encryptionKey: string,
    satoshis: number,
    publicKey: string
  ): Promise<void> {
    console.log('Storing record in MongoDB:', {
      fileUrl,
      encryptionKey,
      satoshis,
      publicKey
    });

    try {
      await this.records.insertOne({
        fileUrl,
        encryptionKey,
        satoshis,
        publicKey
      });
    } catch (error) {
      console.error('Failed to store record:', error);
      throw error;
    }
  }
}
```

---

## 5. Create the decodeOutputs util

This helper parses the BEEF payloads retrieved from your Topic Manager. Note that a very similar function will need to be set up for the frontend:

```ts
// key-server/src/utils/decodeOutputs.ts
import { Transaction, PushDrop, Utils } from '@bsv/sdk'
import { DecodedOutput } from 'src/types.js'

type FieldKey = keyof DecodedOutput

export async function decodeOutput(
  beef: number[],
  outputIndex: number,
  fields: FieldKey[]
): Promise<DecodedOutput> {
  const decodedTx = Transaction.fromBEEF(beef)
  const outputs = decodedTx.outputs[outputIndex]
  const decoded = PushDrop.decode(outputs.lockingScript)
  const data = decoded.fields

  const record: DecodedOutput = {
    txid: decodedTx.id('hex'),
    outputIndex,
  }
  
  // loop through each of the keys and add the ones that match the requisite fields
  for (const key of fields) {
    switch (key) {
      case 'fileUrl':
        record.fileUrl = Utils.toUTF8(Utils.toArray(data[0]))
        break
      case 'name':
        record.name = Utils.toUTF8(Utils.toArray(data[1]))
        break
      case 'description':
        record.description = Utils.toUTF8(Utils.toArray(data[2]))
        break
      case 'satoshis':
        record.satoshis = parseInt(data[3].toString(), 10)
        break
      case 'creatorPublicKey':
        record.creatorPublicKey = Utils.toUTF8(Utils.toArray(data[4]))
        break
      case 'size':
        record.size = parseInt(data[5].toString(), 10)
        break
      case 'retentionPeriod':
        record.retentionPeriod = parseInt(
          Utils.toUTF8(Utils.toArray(data[6])),
          10
        )
        break
      case 'coverUrl':
        record.coverUrl = Utils.toUTF8(Utils.toArray(data[7]))
        break
      case 'txid':
      case 'outputIndex':
      default:
        break
    }
  }
  return record
}

export async function decodeOutputs(
  outputs: Array<{ beef: number[]; outputIndex: number }>,
  fields: FieldKey[]
): Promise<DecodedOutput[]> {
  return Promise.all(
    outputs.map(({ beef, outputIndex }) =>
      decodeOutput(beef, outputIndex, fields)
    )
  )
}
```

---

## 6. Configure Express & middleware

In `key-server/src/index.ts`, set up your app skeleton:

```ts
// key-server/src/index.ts
import express, { Express, Request, Response, NextFunction } from 'express'
import bodyParser from 'body-parser'
import dotenv from 'dotenv'
import { createAuthMiddleware } from '@bsv/auth-express-middleware'
import { createPaymentMiddleware } from '@bsv/payment-express-middleware'
import { getWallet } from './utils/walletSingleton.js'
import { MongoClient } from 'mongodb'
import { KeyStorage } from './KeyStorage.js'
import { LookupResolver, P2PKH, PrivateKey, PublicKey, StorageDownloader, StorageUtils, SymmetricKey, Utils } from '@bsv/sdk'
import { decodeOutputs } from './utils/decodeOutputs.js'
import { DecodedOutput } from './types.js'
import { randomBytes } from 'crypto'

(global.self as any) = { crypto }

dotenv.config()

const app: Express = express()
const PORT = process.env.PORT || 3000
const MONGO_URI = process.env.MONGO_URI as string
const DATABASE_NAME = process.env.DATABASE_NAME as string
const SERVER_PRIVATE_KEY = process.env.SERVER_PRIVATE_KEY as string
const NETWORK_PRESET = process.env.NETWORK_PRESET as 'local' | 'mainnet' | 'testnet'

// Core middleware
app.use(bodyParser.json())
app.use((req, res, next: NextFunction) => {
  res.header('Access-Control-Allow-Origin', '*')
  next()
})

// CORS Headers
app.use((req, res, next: NextFunction) => {
  res.header('Access-Control-Allow-Origin', '*')
  res.header('Access-Control-Allow-Headers', '*')
  res.header('Access-Control-Allow-Methods', '*')
  res.header('Access-Control-Expose-Headers', '*')
  res.header('Access-Control-Allow-Private-Network', 'true')
  if (req.method === 'OPTIONS') {
    res.sendStatus(200)
  } else {
    next()
  }
})

const wallet = await getWallet()

// Auth middleware: enforces authenticated identityKey
app.use(createAuthMiddleware({
  wallet,
  allowUnauthenticated: false,
  logger: console,
  logLevel: undefined
}))

// Payment middleware: all endpoints free for now
app.use(createPaymentMiddleware({
  wallet,
  calculateRequestPrice: async () => 0
}))

// MongoDB and KeyStorage setup
let dbClient: MongoClient
let keyStorage: KeyStorage


app.listen(PORT, async () => {
  console.log(`Server is running on port ${PORT}`)

  try {
    dbClient = new MongoClient(MONGO_URI)
    await dbClient.connect()
    console.log('Connected to MongoDB')

    const db = dbClient.db(DATABASE_NAME)
    keyStorage = new KeyStorage(db)
  } catch (error) {
    console.error('Failed to connect to MongoDB:', error)
    process.exit(1)
  }
})
```

### What's happening:
1.  Load environment variables
2.  Parse JSON bodies & enable CORS
3.  Apply authentication & payment guards
4.  Connect to MongoDB and initialize `KeyStorage`

---

## 7. Add the `/submit` route

Finally, handle uploads by validating, decoding, and storing metadata:

```ts
// key-server/src/index.ts (add after storage setup)

app.post('/submit', async (req: Request, res: Response): Promise<any> => {
  try {
    const { fileUrl, encryptionKey, satoshis, publicKey } = req.body;

    if (!fileUrl || !encryptionKey || !satoshis || !publicKey) {
      return res.status(400).json({ message: 'Missing required fields' })
    }

    console.log(`File URL: ${fileUrl}`)
    // checking if the file is on UHRP
    if (!StorageUtils.isValidURL(fileUrl)) {
      return res.status(400).json({ message: `Invalid file Url: ${fileUrl}` });
    }

    const storageDownloader = new StorageDownloader()

    let resolvedUrl
    for (let attempt = 1; attempt <= 6; attempt++) {
      try {
        resolvedUrl = await storageDownloader.resolve(fileUrl)
        if (resolvedUrl) {
          break
        }
      } catch (error) {
        console.log(`Download attempt ${attempt} failed:`, error)
        if (attempt < 12) {
          await new Promise((resolve) => setTimeout(resolve, 5000))
        } else {
          throw new Error(`Download failed after ${attempt} attempts`)
        }
      }
    }

    if (!resolvedUrl || resolvedUrl.length === 0) {
      return res.status(404).json({ message: 'File not found on UHRP', fileUrl });
    }

    // checking the encryption
    const uhrpFile = await storageDownloader.download(fileUrl)
    const encryptedDataArray = uhrpFile.data
    if (!encryptedDataArray || encryptedDataArray.length === 0) {
      return res.status(400).json({ message: 'Downloaded file is empty' })
    }

    try {
      // ensure that the file decrypts
      const symmetricKey = new SymmetricKey(encryptionKey, 'hex')
      const decryptedFile = symmetricKey.decrypt(encryptedDataArray)
      console.log('File decrypted successfully, length:', decryptedFile.length)
    } catch (error) {
      console.error('Decryption error:', error)
      return res.status(400).json({ message: 'Failed to decrypt file' })
    }


    // checking if the file is properly stored in the backend server
    for (let attempt = 1; attempt <= 6; attempt++) {
      const lookupResolver = new LookupResolver({ networkPreset: NETWORK_PRESET })
      try {
        const check = await lookupResolver.query({
          service: 'ls_demo_market',
          query: {
            type: 'urlCheck',
            value: { fileUrl }
          }
        })

        if (check.type !== 'output-list') throw new Error('Lookup answer must be an output-list')

        const fields: (keyof DecodedOutput)[] = [
          'txid',
          'outputIndex'
        ]
        const result = await decodeOutputs(check.outputs, fields)

        if (result.length >= 0) {
          console.log(`Lookup success in ${attempt} attempts:`, result)
          break
        }
      } catch (error) {
        console.log(`Lookup attempt ${attempt} failed`)
        break
      }

      if (attempt < 6) {
        await new Promise((resolve) => setTimeout(resolve, 5000))
      } else {
        throw new Error(`Lookup failed after ${attempt} attempts`)
      }
    }

    // finally store the file url and its key in the database
    await keyStorage.storeRecord(fileUrl, encryptionKey, satoshis, publicKey)

    return res.status(200).json({ message: 'File metadata saved successfully' })
  } catch (error) {
    console.error('Error submitting file data:', error)
    res.status(500).json({ message: 'Failed to save file metadata' })
  }
})
```

### Key steps in `/submit`:

1.  Validate request body & UHRP URL   
2.  Confirm admission on-chain via Lookup Service
3.  Store record in MongoDB

---

## 8. Run the Key Server

From your `key-server/` directory, start the server and confirm it boots without errors:

```bash
cd key-server
npm run start
```

You should see logs similar to:

```
Server is running on port 3000
Connected to MongoDB
```

If those messages appear and no errors are thrown, your Key Server is correctly wired and ready to accept `/submit` requests from the frontend.

You've now fully configured your Key Server to receive upload submissions, verify on-chain commitment, and persist file metadata. Next: build the Upload Files page on your frontend to call this endpoint!