A supplier import rarely fails on the content of the data. It fails on the pipe: a file read while it was still being written, a password that expired on a Sunday evening, a delta taken for a full catalogue. This article covers that layer — FTP, SFTP, EDI PRICAT, scheduling, acknowledgements, recovery after failure. Attribute mapping and value normalisation are a separate subject, which deserves an article of its own.
Transport is a layer of its own
Two questions are often handled together although they have neither the same failure causes nor the same owners. The first is semantic: what does the supplier’s “PDS_NET” column mean, in which unit, and which attribute should it land in? The second is logistical: how does the file reach you, how do you know it is complete, and what happens when it doesn’t arrive? A perfect conversion rule will say nothing about a file truncated because the connection dropped. Transport has its own failure modes: it decides whether your imports hold up across fifty suppliers or demand human supervision every morning.
FTP, FTPS, SFTP: three acronyms, two protocols
All three circulate as synonyms in exchanges with suppliers. They are not, and the confusion is paid for in lost configuration days.
FTP and FTPS
Legacy FTP opens two separate connections: a command channel on port 21 and a data channel negotiated apart, in active or passive mode. That separation is the source of half the firewall incidents, and FTP carries credentials and content in clear text. FTPS adds a TLS layer to the same protocol, in explicit or implicit mode: that settles confidentiality, not the two-channel gymnastics.
SFTP
SFTP has no relation to FTP despite its name: it is a subsystem of SSH, on a single connection, port 22 by default. One encrypted session carries both commands and data, which simplifies firewall rules, and it accepts public-key authentication — decisive in production, where a password shared between three people always ends up expiring at the worst moment. If the supplier lets you choose, choose this one; an inherited clear-text FTP is debt that will not survive your next security audit.
Fetching a file without cutting it in half
The most frequent trap with file drops has nothing to do with the protocol: it is the race between the writer and the reader. The supplier pushes a 180 MB catalogue, collection fires during the transfer, and you read a syntactically valid file missing half its content: no error is raised and six thousand references disappear from the catalogue. Three conventions prevent it.
- Atomic writing — the supplier uploads under a temporary name (
catalogue.csv.part) then renames tocatalogue.csvonce the transfer is done. The rename is instantaneous: the file under the expected name is, by construction, always complete. - The trigger file — failing that, the supplier drops a second, empty file (
catalogue.ok) after the first. Collection only fires on sight of the trigger, never on the catalogue alone. - Size stability — without supplier cooperation, the collector observes the size across two spaced readings and only takes the file if it has not moved. That is a safety net, not a guarantee.
One last reflex: do not delete a file after reading it, move it to a dated archive. Replaying six weeks of history from the archive takes an hour; asking fifteen suppliers for those files again takes a month, and half of them no longer have them.
EDI: a message format as much as a pipe
EDI is often presented as “FTP, but more complicated”. That is a misreading: FTP carries files whose content does not concern it, EDI defines the grammar of messages, their sequence and their replies. It answers a question FTP ignores: what must the recipient reply, and in what form?
Anatomy of an EDIFACT PRICAT
PRICAT (Price/Sales Catalogue) is the UN/EDIFACT price catalogue message, the one a distributor receives from its suppliers. An interchange is a sequence of three-letter segments wrapped in two envelopes: UNB … UNZ for the interchange, whose header carries the parties’ identifiers and a unique control reference that makes it possible to detect a duplicate send; UNH … UNT for the message itself, declaring its type, the directory version used (D.96A, D.01B…) and the controlling agency. Between the two come the document identification and the parties (BGM, DTM, NAD, the latter identified by their GLN), then the body: item line and GTIN (LIN), additional identifiers (PIA), descriptions (IMD), measurements (MEA), prices and currency (PRI, CUX).
The directory version is not a cosmetic detail: two suppliers can send a perfectly compliant PRICAT and expect different readings because they do not refer to the same one. On top of that come sector guides — EANCOM, the EDIFACT subset maintained by GS1, narrows segment usage for retail, and each retailer publishes its own on top.
The other messages in the cycle
PRICAT never lives alone: the commercial cycle is also written with ORDERS (purchase order), ORDRSP (order response), DESADV (despatch advice) and INVOIC (invoice), on the same channel and with the same partner. On the North American side, the ANSI X12 standard covers the same needs with different numbering: 832 for the catalogue, 850 for the order.
CONTRL and APERAK: the two acknowledgements
This is the most misunderstood point in EDI, and the one that makes the difference in operations. CONTRL is the technical acknowledgement: it says “I received the interchange, its syntax is correct”, or flags the faulty segment — it says nothing about the meaning of the message. APERAK is the application acknowledgement: “I received, I understood, and here is what my application did with it”, accepted, accepted with reservations, or rejected with a business error code.
A positive CONTRL therefore tells you nothing useful about substance. And a feed that emits no acknowledgement leaves the supplier in the dark: it keeps publishing without knowing that its last three hundred lines are rejected every week. Emitting a usable reply, even outside strict EDI, is what turns an import into an exchange.
Scheduling: frequency, window and time zone
Frequency follows the actual publication rhythm, not an ambition. Polling a server fed once a week every hour drowns the log in empty readings; conversely, a daily collection on a price feed published three times a day has you selling at a stale rate.
The window is the interval during which the file is supposed to be available. It serves to alert as much as to trigger: if nothing has arrived by the end of the window, someone must know. An import that finds nothing is a failed import, not a successful import without data.
The time zone takes its revenge twice a year: a supplier publishing “at 10 pm” publishes at 10 pm where it is, and at the daylight-saving switch the window shifts. Write schedules in an explicit time zone, and stagger collections between suppliers by a few minutes — a network outage at exactly midnight will not then fail fifty feeds at once.
Recovering after a failure
An automated feed is judged on what it does when things go wrong.
- Idempotency — the collector records a fingerprint of every file processed (name, size, checksum) and, on the EDI side, the interchange control reference. A file republished identically is recognised and ignored.
- Step-level retry — collection, parsing and writing are traced separately. When the write fails, you replay the write; you do not go back to a connection to the server, which may no longer hold the file.
- Quarantine — an unreadable file goes to a dedicated directory with the reason for refusal, rather than being retried indefinitely.
- The alert threshold — a few rejected lines are handled as they come; a third of the file rejected is a format change on the supplier side, and must wake someone the same day.
Traps in the transport layer
- A delta taken for a full file — some suppliers send the whole catalogue, others only creations and changes, and some alternate without warning. The behaviour is chosen per feed, never inferred from the file. Corollary: is a reference that drops out of the file discontinued, out of stock, or simply forgotten? Without an explicit rule the catalogue empties itself; the reasonable default is to mark inactive and alert.
- File names that move — going from
catalogue.csvtocatalogue_2026.csvis enough for the collector to find nothing. A tolerant name pattern, together with an alert on files present but unrecognised, settles the case.
What Pixee PIM handles on the transport side
The Pixee PIM import module collects catalogues from FTP, SFTP, direct URL (HTTP) or email attachment (IMAP), on top of manual upload — in CSV, Excel, XML and JSON. EDI messages (PRICAT) are not read natively: they go through an upstream conversion into one of these formats. For each feed you describe the transport and the schedule — access, file name pattern, source format, daily, weekly or custom frequency. The system detects new files, keeps a per-import report and retains the full history: enough to replay a feed or to answer a supplier disputing a delivery. Validating product identifiers on the way in is the job of the EAN Manager module, and the standardised feeds of grocery retail follow a neighbouring logic, described in our article on GDSN / GS1 synchronisation.
A legacy EDI exchange does not have to be dismantled to open an extra channel: that is the principle behind the Sage X3 integration, where the PIM sits alongside the existing feed. Volumes depend on the plan — 2 suppliers and one automated import during the 30-day free trial, 5 suppliers and 150 imports per month on Starter, 25 suppliers with unlimited imports on Growth; the detail is on the pricing page.
Frequently asked questions
Should you require EDI from a supplier who already delivers a correct CSV?
No, as long as the CSV arrives reliably and the supplier stays responsive when something goes wrong. EDI brings a standardised grammar and an acknowledgement cycle, which is only worth something when volume or the number of partners makes manual follow-up impossible. Imposing EDIFACT on a supplier who does not do it amounts to making them fund a service provider for a problem you do not have.
What collection frequency should you pick for a price feed?
The one at which the supplier actually publishes, not the one you would like. The last modified date of the files dropped over two or three weeks gives the effective rhythm. A collection more frequent than publication only fills the log and masks genuine missing files.
What should you do when an import found nothing at the expected time?
Treat it as an anomaly, not as a success. An availability window combined with an alert separates the two cases: the file is late, or it is not coming. Without that distinction an import chain can stay silent for weeks while the catalogue ages.
Can you replay a past import without duplicating data?
Yes, on two conditions: keep source files in a dated archive rather than deleting them after reading, and write idempotently — records are matched on a stable identifier then updated, never appended blindly. Replaying then becomes a risk-free operation.
Automate the collection of your supplier feeds
FTP, SFTP, URL, email — scheduling, per-import report and full history.
Start automating