File Download and Upload with TSOA (Hack)

File Download and Upload with TSOA (Hack)

TSOA provides great flexibility for the task to create REST APIs for web services that conform to the OpenAPI specification. Unfortunately if your service consumes or produces files there is no easy way to setup service functions with TSOA yet. This article tries to provide an example that shows how files can be uploaded or downloaded from a REST service with TSOA, Aurelia and multer.


Upload and Downloading files with OAI

Uploading or downloading files from a REST web service is a common feature.

  • upload images (for example as profile image of a user)
  • upload a sheet with data that the service should process
  • download generated PDF documents with statistics or generated letters
  • download zipped files (for example previously selected songs the user want to download)

The Open API Specification specifies uploading and downloading of files over a REST API. TSOA can generate OpenApi compatible REST services but file handling is a week spot. There are open pull requests for file upload and download but in the meantime you have to find a solution by yourself.

Maybe the following solution will help.

Uploading files

How it works

In short words we put a file in a HTTP request and read it back in the API from the request by hand. To make things easier we use the middleware multer to handle the file streaming from client to server. Since TSOA generates the service routes but do not know file handling at the moment we have to overwrite the generated route with some data annotations.

Lets look at the example.

At the Server side

Here is an example of a controller that contains a service function that consumes a file: uploadImageFile

import { Post, Request, Response, Route, SuccessResponse, Tags } from 'tsoa'

import * as express from 'express'
import * as multer from 'multer'
import { StatusError } from '../models/StatusError'

@Route('files')
export class FilesController {

  @Tags('File')
  @SuccessResponse('204', 'successful')
  @Response('400', 'invalid file supplied')
  @Post('uploadFile')
  public async uploadImageFile (@Request() request: express.Request): Promise {
    await this.handleFile(request, 'imageFile', 'zip', 'tmp', 'img.zip')
    // file will be in request.imageFile, it is a buffer
  }

  private async handleFile (request: express.Request,
                            requestField: string,
                            filterExt: string,
                            storePath: string,
                            fileName?: string): Promise {
    const fileFilter = (req: Express.Request, file: Express.Multer.File,
                        callback: (error: Error | null, acceptFile: boolean) => void) => {
      // accept image only
      if (!file.originalname.match(`^(.*\\.((${filterExt})$))

TSOA provides great flexibility by defining REST service APIs. Unfortunately if your service consumes or produces files there is no easy way to setup service functions yet. This article tries to provide an example that shows how files can be uploaded or downloaded from a REST service with TSOA, Aurelia and multer.


Upload and Downloading files with OAI

Uploading or downloading files from a REST web service is a common feature.

  • upload images (for example as profile image of a user)
  • upload a sheet with data that the service should process
  • download generated PDF documents with statistics or generated letters
  • download zipped files (for example previously selected songs the user want to download)

The Open API Specification specifies uploading and downloading of files over a REST API. TSOA can generate OpenApi compatible REST services but file handling is a week spot. There are open pull requests for file upload and download but in the meantime you have to find a solution by yourself.

Maybe the following solution will help.

Uploading files

How it works

In short words we put a file in a HTTP request and read it back in the API from the request by hand. To make things easier we use the middleware multer to handle the file streaming from client to server. Since TSOA generates the service routes but do not know file handling at the moment we have to overwrite the generated route with some data annotations.

Lets look at the example.

At the Server side

Here is an example of a controller that contains a service function that consumes a file: uploadImageFile )) { return callback(new Error(‚Only zip files are allowed!‘), false) } callback(null, true) } // store at disk const storage = multer.diskStorage({ destination: (req, file, callback) => { callback(null, `${storePath}/`) }, filename: (req, file, callback) => { callback(null, fileName ? fileName : file.originalname) } }) const upload = multer({ storage, fileFilter }) const multerSingle = upload.single(requestField) try { await new Promise(async (resolve, reject) => { try { multerSingle(request, undefined, async (error) => { if (error) { reject(error) } resolve() }) } catch (error) { reject(error) } }) } catch (error) { throw new StatusError(400, ‚invalid upload‘, error.toString()) } } }

There is only this one upload function in this controller. You can put other functions beside it without causing damage. For our example we wanted to keep it clean and readable.

The method handleFile will upload any given file and store it locally in an directory. For simplicity the example assumes that there is an uploaded zip file that is stored locally as temp/img.zip

The TSOA annotations will generate a REST function with the url http://localhost/files/uploadFile

We can give TSOA custom parameters in the tsoa.json configuration that will be used to generate the route of this URL. So we add the request parameters for multipart file transfer. TSOA will use this definition to generate a compliant OpenAPI description (swagger.json).

{
    "swagger": {
		...
        "specMerging": "recursive",
        "spec": {
          "paths": {
            "/files/uploadFile": {
              "post": {
                "consumes": [
                  "multipart/form-data"
                ],
                "parameters": [
                  {
                    "in": "formData",
                    "name": "imageFile",
                    "required": true,
                    "type": "file"
                  }
                ]
              }
            }
          }
        }
    },
    "routes": {
        ...
    }
}

This configuration expects that the uploaded file is stored in the field imageFile so that our controller can read the file from this field.

That’s it! You can recompile and start your service and it will offer a function where you can upload a single file.

Upload a file from a web client

To upload a file to the running service function we could use the Swagger UI. We can trigger an upload over a custom GUI. We can use Aurelia for a very simple GUI running in a web browser.

<template xmlns="http://www.w3.org/1999/html">
  <div id="upload-image" class="detail-view">
        <input type="file"
          accept=".zip"
          files.bind="selectedImages">

        <a class="run-upload" click.delegate="startUpload()" if.bind="selectedImages.length">
          <img src="../images/01_upload_32x32.png">
        </a>
  </div>
</template>
import * as client from 'backend-client'

import { autoinject } from 'aurelia-dependency-injection'

@autoinject
export class UploadImageFile {
  public selectedImages: FileList

  constructor (
      private fileApi: client.FileApi,
  ) {}

  public startUpload () {
    return this.fileApi.uploadImageFile({ imageFile: this.selectedImages })
      .then(async () => {
          LOG.info('import completed successfull')
        }).catch((error: any) => {
          LOG.error('import failed', error)
        })
  }
}
uploadImageFile(params: {  "imageFile": FileList; }, configuration: Configuration, options: any = {}): FetchArgs {
        // verify required parameter "imageFile" is set
        if (params["imageFile"] == null) {
            throw new Error("Missing required parameter imageFile when calling uploadImageFile");
        }
        const files = params["imageFile"]
        const baseUrl = `/files/uploadFile`;
        let urlObj = url.parse(baseUrl, true);
        let fetchOptions: RequestInit = Object.assign({}, { method: "POST" }, options);


        let contentTypeHeader: Dictionary = {};
        // DO NOT provide ANY Content-Type Header!!! Let the browser take care of setting the content-type!
        contentTypeHeader = { "accept": "application/json" };

        if (contentTypeHeader) {
            fetchOptions.headers = Object.assign({}, contentTypeHeader, fetchOptions.headers);
        }
        // we send the binary file appended to a form data
        const formData = new FormData()
        for (let i = 0; i < files.length; i++) {
            formData.append("imageFile", files[i]);
        }
        fetchOptions.body = formData;

        return {
            url: url.format(urlObj),
            options: fetchOptions,
        };
    },

Our small example GUI opens a file chooser dialog that lets the user select a zip file from his computer. The upload will start when he presses the upload button in the GUI.

To upload the file a client (client.FileApi) is used. You can generate a suitable client with Swagger Codegen. But you have to overwrite the generated function for our uploadImageFile function with the example code.

The custom code puts the zip file in a form data and in the field imageFile that our service function will use for reading the files content.

Downloading files

How it works

This solution works similar to the upload example but we have to have to hack our way through the TSOA code. On the server side we need a physical file that is feed to multer and on the client side read it back into the browsers memory. We have to use another hack to let the browser think he is downloading the file he already stored in his memory. So this solution get a little bit dirty but show how it could work.

At the Server side

Again we put a download function into a controller: our FilesController.downloadImageFile()

We add a custom route description for the new download function. We describe that the new URL http://localhost/files/download/image will produce a zip file.

import { Post, Request, Response, Route, SuccessResponse, Tags } from 'tsoa'

import * as express from 'express'
import * as multer from 'multer'
import { StatusError } from '../models/StatusError'

@Route('files')
export class FilesController {

  @Tags('File')
  @SuccessResponse('200', 'successful')
  // @Produces('application/zip, application/octet-stream') // hopefully this will work in the near future
  @Get('download/image')
  public async downloadImageFile (): Promise {
    const absPath = await service.generateImageFile()

    return FileResult.newInstance({ path: absPath })
  }
}
{
    "swagger": {
		...
        "specMerging": "recursive",
        "spec": {
          "paths": {
            "/files/download/image": {
              "get": {
                "produces": [
                  "application/zip, application/octet-stream"
                ]
              }
            },
          }
        }
    },
    "routes": {
        ...
    }
}

We are not done yet. To let TSOA generate routes that handle file downloads we have to rewrite some TSOA code. TSOA does not know the file type yet. To let TSOA handle files when generating files we have to copy the existing code and add our own route handling for files. It should be ok since we use the altered code for file download functions only anyway.

For this matter the classes FileResult are created. If TSOA finds a file result it adds file handling into the the generated routes code.

import { authenticateMiddlewareExt, getValidatedArgsExt, promiseHandlerExt } from './utils/tsoaUtils'

// ...

// we need content-disposition to send the filename to the client
res.header('Access-Control-Expose-Headers', 'Content-Disposition,Content-Type,Content-Length')

 // register the new custom route BEFORE the RegisterRoutes() call
app.get('/v1/download/image',
  authenticateMiddlewareExt([{ 'name': 'sample_auth', 'scopes': ['DOWNLOAD'] }]),
  (request: any, response: any, next: any) => {
    const args = {
    }

    let validatedArgs: any[] = []
    try {
      validatedArgs = getValidatedArgsExt(args, request)
    } catch (err) {
      return next(err)
    }

    const controller = new FilesController()

    const promise = controller.downloadImageFile.apply(controller, validatedArgs)
    promiseHandlerExt(controller, promise, response, next).catch(async (error) => LOG.error(error))
  })

RegisterRoutes(app)
/* tslint:disable */
import { Readable } from 'stream';
import { Controller, ValidateParam, FieldErrors, ValidateError, TsoaRoute } from 'tsoa';
import { expressAuthentication } from '../authentication'

export class FileResult {
  public path?: string;
  public data?: any;
  public filename?: string;

  constructor (initial: FileResult = {}) {
    this.path = initial.path
    this.data = initial.data
    this.filename = initial.filename
  }

  public static newInstance (initial: FileResult = {}): FileResult {
    return new FileResult(initial)
  }
}
const models: TsoaRoute.Models = {

}

export function authenticateMiddlewareExt(security: TsoaRoute.Security[] = []) {
  return (request: any, response: any, next: any) => {
    let responded = 0;
    let success = false;
    for (const secMethod of security) {
      expressAuthentication(request, secMethod.name, secMethod.scopes).then((user: any) => {
        // only need to respond once
        if (!success) {
          success = true;
          responded++;
          request['user'] = user;
          next();
        }
      })
        .catch((error: any) => {
          responded++;
          if (responded == security.length && !success) {
            response.status(401);
            next(error)
          }
        })
    }
  }
}

export function promiseHandlerExt (controllerObj: any, promise: any, response: any, next: any) {
  return Promise.resolve(promise)
    .then((data: any) => {
      let statusCode
      if (controllerObj instanceof Controller) {
        const controller = controllerObj as Controller
        const headers = controller.getHeaders()
        Object.keys(headers).forEach((name: string) => {
          response.set(name, headers[name])
        });

        statusCode = controller.getStatus()
      }

      if (data) {
        if (data instanceof FileResult) {
          // send file name in header
          let filename = data.filename ? data.filename : (data.path ? data.path.replace(/^.*[\\\/]/, '') : undefined)
          if (filename) {
            response.set('Content-Disposition', `filename="${filename}"`)
          }
          
          // read file from path and send it (not response.download())
          if (data.path) {
            response.status(statusCode | 200)
            data.filename ? response.sendFile(data.path, data.filename) : response.sendFile(data.path)
          } else if (data.data) {
            if (data.data instanceof Readable) {
              response.status(statusCode | 200)
              data.data.pipe(response)
            } else {
              response.status(statusCode | 200).send(data.data)
            }
          } else {
            response.status(statusCode || 404).end()
          }
        } else {
          response.status(statusCode | 200).json(data)
        }
      } else {
        response.status(statusCode || 204).end()
      }
    })
    .catch((error: any) => next(error))
}

export function getValidatedArgsExt(args: any, request: any): any[] {
  const fieldErrors: FieldErrors = {};
  const values = Object.keys(args).map((key) => {
    const name = args[key].name;
    switch (args[key].in) {
      case 'request':
        return request;
      case 'query':
        return ValidateParam(args[key], request.query[name], models, name, fieldErrors);
      case 'path':
        return ValidateParam(args[key], request.params[name], models, name, fieldErrors);
      case 'header':
        return ValidateParam(args[key], request.header(name), models, name, fieldErrors);
      case 'body':
        return ValidateParam(args[key], request.body, models, name, fieldErrors, name + '.');
      case 'body-prop':
        return ValidateParam(args[key], request.body[name], models, name, fieldErrors, 'body.');
    }
  });
  if (Object.keys(fieldErrors).length > 0) {
    throw new ValidateError(fieldErrors, '');
  }
  return values;
}

This example adds a download function to a REST API of a web service.

Download a file to a web client

You can call the new download function over the Swagger UI.

Alternatively you can include the download into a web client running in a browser. Again we can use Aurelia for a very simple GUI client.

<template xmlns="http://www.w3.org/1999/html">
  <div id="download-image" class="detail-view">
        <button class="download-button" click.trigger="startDownload()"><img src="../images/01_download_32x32.png"><span>Download</span></button>
  </div>
</template>
import * as client from 'backend-client'

import { autoinject } from 'aurelia-dependency-injection'

@autoinject
export class DownloadImageFile {
  public selectedImages: FileList

  constructor (
      private fileApi: client.FileApi,
  ) {}

  public startDownload () {
    // download image file
    this.fileApi.downloadImageFile().then(async (result) => {
      const resp: Response = result.data as Response

      return this.downloadFile(resp, 'downloaded.zip')
    }).catch((error: any) => LOG.error(error)) 
  }

  private async downloadFile (response: Response, defaultName: string): Promise {
    let filename = defaultName

    // get filename from header
    response.headers.forEach((name: string, val: string) => {
      if (val.toLowerCase() === 'content-disposition') {
        const filenameRegex = /filename[^;=\n]*=((['"]).*?\2|[^;\n]*)/
        const matches = filenameRegex.exec(name)
        if (matches !== null && matches[1]) {
          const tmp = matches[1].replace(/['"]/g, '')
          if (tmp) {
            filename = tmp
          }
        }
      }
    })

    // read data from stream
    return response.blob().then(async (blob) => URL.createObjectURL(blob))
    .then(async (url) => {
      // create link for temporary URL and klick it
      const link = document.createElement('a')

      link.href = url
      link.download = filename
      link.click()
    })
  }
}
export const FileApiFetchParamCreator = {
    downloadImageFile(configuration: Configuration, options: any = {}): FetchArgs {
        const baseUrl = `/files/download/image`;
        let urlObj = url.parse(baseUrl, true);
        let fetchOptions: RequestInit = Object.assign({}, { method: "GET" }, options);


        let contentTypeHeader: Dictionary = {};
        if (contentTypeHeader) {
            fetchOptions.headers = Object.assign({}, contentTypeHeader, fetchOptions.headers);
        }

        return {
            url: url.format(urlObj),
            options: fetchOptions,
        };
    },
// ...
export const FileApiFp = {
    downloadImageFile(configuration: Configuration, options: any = {}): (fetch?: FetchAPI, basePath?: string) => Promise {
        const fetchArgs = FileApiFetchParamCreator.downloadImageFile(configuration, options);
        return (fetch: FetchAPI = httpClient.fetch, basePath: string = BASE_PATH) => {
            return fetch(basePath + fetchArgs.url, fetchArgs.options).then((response) => {
                if (response.status >= 200 && response.status < 300) {
// we put the whole response into data
                    return { data: response };
                } else {
                    throw response;
                }
            });
        };
    },
// ...

Again the used client can be generated with Swagger Codegen but the generated implementation should be replaced with the given example code. The code handles the services response by returning it the complete response to the calling function.

When the user clicks the download button in the GUI the client calls the downloadImageFile at the web service and waits for the response. If the client gets the response it first reads the file name of the returned file. Next it applies a hack to let the browser handle the file as download. The client loads the file puts it into a link and clicks the created link all within code and memory. This causes the browser to handle the file as a typical download file and ask the user if to open or where to store the file. It’s a little trick to avoid writing different code for different browsers.

Schreibe einen Kommentar

Deine E-Mail-Adresse wird nicht veröffentlicht. Erforderliche Felder sind mit * markiert.