Blog

SFTP integration using SFTP Uploader app

It is possible to upload invoices to an SFTP server and have them picked up from there automatically. It provides secure file system upload over a secure channel. Tradeshift provides a free account to a server where you can upload your invoices in a variety of formats.


See currently supported formats on our integration page.

SFTP stands for ‘SSH File Transfer Protocol’, it is not related to FTP except that it also transfers files and has a similar command set for users. If you are not familiar with SFTP it is recommended to find out more about it before continuing reading this page. There is plenty of information about SFTP available on the web.

In order to use this integration method you have to have an accounting or ERP software which is able to send documents over SFTP or an SFTP client (there are plenty of free SFTP clients available). You have also to activate the ‘SFTP Uploader‘ app from the Tradeshift apps.

To integrate your ERP, accounting system or SFTP client with Tradeshift, please, follow the following steps:

1. Go to the SFTP Uploader app

2. Generate OpenSSH RSA or DSA public and private keys. (see below)

3. Paste generated public key into the text area

4. Press ‘Save’ button to save the key and create an account at the SFTP server. If the key is invalid or in a wrong format an error message will show up.

5. Copy the SFTP hostname, port and username shown below the text area

SFTP App settings

6. Use SFTP hostname, port, username and your private key to configure your ERP, accounting software or SFTP client to connect to the SFTP server. SFTP server doesn’t need password.

7. That’s it! You are ready to send invoices from your system or SFTP client to the SFTP server. Try to connect and put a document into the ‘outbox’ directory of the server.

Latest changes

    • It is possible to enable more formats for dispatch result messages. When a file was dispatched, you can find a dispatch result file (an XML file which has the same name as the original file followed by ‘.dispatch.xml’) in the ‘/sent’ or ‘/dispatchfailed’ directory. This is the default Tradeshift dispatch result file. More result formats can be enabled in the ‘Advanced’ tab of the SFTP app. Currently we support only ‘CONTRL’ which is EDIFACT format control message, but more formats will follow.

 

  • (S)FTP(S) usernames became Base64 encoded in order to reduce their length from 32 to 22 characters. For example, if your previous username looked like this: 877308069fd74fed822f3c8afd5e3579, it will become similar to h3MIBp/XT+2CLzyK/V41eQ. All characters like ‘/’, ‘+’ are valid. We still support 32 characters long usernames, so there is no need of changing your existing configuration.

What is OpenSSH?

OpenSSH provides end-to-end encrypted channel to facilitate files exchange. It’s a more secure replacement of applications such as telnet, rlogin, and ftp. Unlike these legacy applications, OpenSSH never passes anything (including username and password) over the wire in unencrypted form, and provides host authentication, to verify that you really are talking to the system that you think you are and that no one else can take over that session. OpenSSH is a suite of tools to help secure your network connections.
It provides a strong authentication and closes several security holes (e.g., IP, routing and DNS spoofing). It has improved privacy and all communications are automatically and transparently encrypted.

OpenSSH is available from http://www.openssh.org.

 

Tradeshift’s SFTP server directory structure

Dir name Description
outbox Directory used to put files to be processed by Tradeshift. Read more about supported document formats. Uploaded files MUST have unique names.
NOTE: This server implements different semaphore mechanism to allow you to signal that an upload is completed. You MUST employ one of these methods – if you do not, the server will wait indefinitely for the upload to complete.

First semaphore mechanism: Using .ok notation
After succesfully uploading a file <filename> (e.g. invoice-1.xml), either: Upload a second possibly empty file named <filename>.ok (e.g. invoice-1.xml.ok), or: Rename from <filename> to <filename>.ok

Second semaphore mechanism: Using .tmp notation
Upload the file as <filename>.tmp (e.g. invoice-1.xml.tmp), and after successfull completion, rename the file to <filename> (e.g. invoice-1.xml).

Third semaphore mechanism: Using .filename notation
Upload the file as .<filename> (e.g. .invoice.xml), and after successfull completion, rename the file to <filename> (e.g. invoice-1.xml).Only one document (e.g. invoice, credit note) per one file is supported.

This server supports attachments to documents. Attachments are files which are attached to the target document. Attachments can be send only together with the original document. It is not possible to add attachments to already uploaded document.

Attachments file names MUST be unique for the target document.
If sending a document with attachments they MUST be uploaded to the server BEFORE the signal about completed upload of the target document (read about semaphore semantics above). Only ONE signal about completed upload should be made: the signal about the target document upload. Do NOT signal completion of attachment files upload.
If there are no attachments to the target document or they do not confirm to the document attachments naming convention, the target document is send without attachments.
The attachment file names to the target document MUST confirm to ONE of the following rules:

attachment[documentfilename]attachmentfilename
Name starts with the word ‘attachment’ and the file name of the target document inside the square brackets and followed by the file name of the attachment itself.
E.g. the attachment which original name is receipt.pdf and has to be attached to invoice which file name is invoice-01.xml must be put to the server with file name attachment[invoice-01.xml]receipt.pdf

attachmentfilename.attachment.documentfilename
Name starts with original attachment’s file name followed by ‘.attachment.’ and ends with the file name of the target document.
E.g. the attachment which original name is receipt.pdf and has to be attached to invoice which file name is invoice-01.xml must be put to the server with file name receipt.pdf.attachment.invoice-01.xml

Attachments are attached to the target document with their original file names (e.g. receipt.pdf). Attachments are REMOVED from the server after the document is processed, no matter successfully or not.

sent Stores successfully dispatched files. Besides, there will be dispatch results file available for each sent document. The dispatch results file has the same name as the original file, but has ‘.dispatch.xml’ appended. More dispatch result formats can be enabled in the ‘Advanced’ tab of the SFTP
failed Stores files which didn’t pass the validation. Besides, there will be validation results file available for each failed validation file. It has the same name as the original file but ‘.failed.xml’ appended.
dispatchfailed Stores files which pass the validation, but couldn’t be dispatched. Besides, there will be dispatch results file available for each failed validation file. It has the same name as the original file but ‘.dispatchfailed.xml’ appended. Dispatch might fail because of invalid e-mail address in the document.
dynamicvalidations Optional directory. You have to activate Dynamic Validations app to have it enabled. It stores definitions of dynamic validation. The dynamic validation definitions are XML files defining fields of documents which should be validated against list of possible values that are dynamically uploaded to the dynamicvalidationchanges directory. The validations may define the sender and the receiver of the document for which the field must be validated. It is particularly valuable for an enterprise which has branches (see Branch Management) app: the different validations can be be defined for each branch by setting the branch as the receiver.All files must have the following mandatory fields:

  • Id – the unique ID of the validation definition
  • ReceiverId – the receiver of the document for which the validation must apply. It is defined by the scheme (for example “GB:VAT”, “DK:CVR”) and the actual value this scheme represents
  • Field – the XPath expression unambiguously defining the field of the document to apply validation to
  • DocumentType – the type of the document which is to be validated. For example: invoice, quote, etc.

Optionally, the SenderId could be defined. In this case the validation is applied only to documents sent by this particular sender.

NOTE: the file name must exactly match to the Id of the validation definition appended by “.xml” string.

Only one validation definition per file is allowed.

Following the SFTP’s server “.ok file” strategy, the file will be uploaded only after an empty file with the same name appended with “.ok” is uploaded to the directory.

dynamicvalidationchanges Optional directory. You have to activate Dynamic Validations app to have it enabled. Directory to upload list of valid values of fields defined by the dynamic validation definitions (see dynamicvalidation directory description). They are uploaded as XML files containing reference (dynamicValidationId) to the correspondent dynamic validation definition uploaded earlier. If the referenced dynamic validation definition does not exist, the uploaded file will be skipped. NOTE: The file name must exactly match to the dynamicValidationId of the validation definition appended by “.xml” string. File will be uploaded only after an empty file with the same name appended with “.ok” string is uploaded. The uploaded files are not viewable and the directory is always empty.
network Optional directory. You have to activate Van accessapp to have it enabled.The directory is used by the external value added networks (VAN) to send documents on behalf of their customers. VAN’s customers don’t have to be registered in Tradeshift. Tradeshift issues a unique ID to each VAN. Contact Tradeshift to request the ID.The VAN ID will appear as a sub-directory of the “/network” directory. This sub-directory works as the “/outbox” directory for the regular users, i.e. documents put into this sub-directory are dispatched. For example, a VAN witch ID is “van01” will have it’s outbox directory in the following location:/network/van01NOTE: you can’t put files into the “/network” directory.
All rules and semaphore mechanisms of the “outbox” directory apply to this directory. Results are put in the regular failed, dispatchfailed and sent folders.

How to generate SSH keys on Windows

You can use PuTTY to generate SSH keys. PuTTY is a number of programs that work with unix-like servers. You can download PuTTY from http://www.putty.org/.

You will need puttygen.exe program to generate keys.
1. Run puttygen.exe. You will see a GUI similar to the one shown below

2. Select ‘SSH-2 RSA’ type, 1024 number of bits in a generated key and press the ‘Generate’ button. Then move your mouse around the blank area to generate some randomness for the key

3. The keys will now be generated. You will now need the OpenSSH public key for pasting into the ‘SFTP upload’ app which you can copy from the ‘key’ area. Copy and paste into the ‘Public key’ of the app

Puttygen GUI

4. Please leave the ‘key passphrase’ field empty and move on to the next step.

5. Save public key by pressing ‘Save public key’ button, input file name ‘sftp_rsa_pub.ppk’. Save private key by pressing ‘Save private key’ button, input file name ‘sftp_rsa.ppk’. You will use the files later when connecting to the SFTP server with an SFTP client

How to generate SSH keys on Unix

Preliminaries
OpenSSH

Generate public/private key pair
1. Run ssh-keygen to create the public/private key pair:
local-host$ ssh-keygen -t rsa
2. This will prompt you for a secret passphrase. Please leave the passphrase field empty and press Enter. If this works right you will get two files called id_rsa and id_rsa.pub in your .ssh directory. You will use them later when connecting to the SFTP server with an SFTP client.

Dispatch results

After the signaling that the upload is complete using one of the semaphore mechanisms, the file is converted, validated, digitally signed (optional) and dispatched to the recipient. All this is done asynchronously and take some time. Usually, it takes less than 1 minute, but in case when digital signing is involved or there is a heavy load of the server the dispatching may take more time. So, give the system 5-10 minutes before escalating dispatching issue to a problem. When the file has been dispatched successfully it is moved to the ‘/sent’ directory. Together with this file there will be a dispatch log file in XML format which has the same name as the original file followed by ‘.dispatch.xml’. This is the default Tradeshift dispatch result format. More result formats can be enabled in the ‘Advanced’ tab of the SFTP/FTP/FTPS app. Currently we support only ‘CONTRL’ which is EDIFACT format control message, but more formats will follow. If the dispatch has failed for some reason, the file is moved to the ‘/dispatchfailed’ directory. The dispatch result files can be found in this directory too.

Test your SFTP connection in Windows

After the key was generated and successfully saved by the ‘SFTP Upload’ app, it would be a good idea to test the connection to SFTP server. For the test purposes you can use WinSCP – a free simple SSH client. You can download it from the WinSCP website http://winscp.net/eng/download.php. Install WinSCP and follow the following steps:

1. Run WinSCP, click on ‘New’ button to start a new SSH session

2. Fill in ‘Host name’, ‘Port number’ and ‘User name’ fields with the values provided in ‘SFTP Upload’ app. Select private key file ‘sftp_rsa.ppk’ you saved when generating the key. Set the file protocol to SFTP. Leave ‘Password’ field empty, SFTP doesn’t use passwords

Setting up WinSCP session

3. Click on ‘Login’ button. If the private key which was provided was saved with a passphrase, you will be asked to provide this passphrase during login

4. If login was successful you will see a commander-style window with list of SFTP server’s directories on one of the sides

SFTP file system

5. Double click on the ‘outbox’ directory to open it. You may get an error message saying something similar to “Server returned empty listing for directory ‘/outbox’”. Ignore it by pressing ‘Ok’ button

6. Create a test invoice. Remember that all your invoices must have unique Invoice Numbers (Invoice ID). Let’s assume, you created an invoice in UBL format with the Invoice Number ’12709′. The name of the invoice must match exactly to the Invoice Number and have ‘xml’ extension: ’12709.xml’

7. Put the invoice into the ‘outbox’ directory of the SFTP server. You can do this by selecting the invoice file in the WinSCP window and press ‘F5′ key to copy the file to the ‘outbox’ directory in the other window. Alternatively, you can just drag the file to the ‘outbox’ and drop it there

8. The file is uploaded to the SFTP server, but it’s not processed yet. To process the file put an empty file with the name which match exactly to the name of the invoice file appended by ‘.ok’: ’12709.xml.ok’. This will trigger Tradeshift to pick up the file and process it

Please be advised that if the files are not uploaded in the right order (1. invoice, 2. semaphore mechanism file with “.ok”) the dispatch engine will not be triggered.

9. The ’12709.xml.ok’ file is deleted. The invoice file ’12709.xml’ is moved to one of the following directories, depending on the validation and dispatch results:

  • sent – in case the invoice passed the validation and was dispatched successfully
  • failed – in case the invoice didn’t pass the validation and was not dispatched
  • dispatchfailed – in case the invoice passed the validation but could not be dispatched (because of the non-existing recipient’s e-mail address)

Read more about the SFTP server directory structure

10. Go to the ‘Documents’ tab of the Tradeshift web platform. If the invoices passed the validation it will appear under the ‘Sales’ subtab

Uploaded invoice in the 'Documents' tab