Good Practice When Validating Input

Having worked for a pharmaceutical company in the past, my first encounter with the concept of validation was the rigorous procedure of testing every aspect of software that was developed or altered. The testing was always extensive because we needed to ensure products and processes fulfilled important regulations to prevent any harm to patients.

Now developing general websites, learning the built-in and customised validation is approaching the subject from an inside-out perspective but still with the user in mind as one of several input data sources.

The main goal of input validation is to ensure only properly formed data is entering the workflow in an information system, preventing malformed data causing malfunction of various components. Input validation should happen as early as possible in the data flow, preferably as soon as the data is received from the external party.

Data from all potentially untrusted sources should be subject to input validation, including not only web clients but backend feeds over extranets from suppliers, partners, vendors and regulators, each of which may be compromised on their own and send malformed data.

Input Validation should not be used as the primary method of preventing attacks but can significantly contribute to reducing their impact if implemented properly.

Input validation should be applied on both syntactical and semantic level. Syntax refers to the structure/form of the code that a specific programming language specifies whilst Semantics deal with the meaning assigned to the symbols, characters and words.

Syntactic validation should enforce correct syntax of structured fields (e.g. date, currency symbol) while semantic validation should enforce correctness of their values in the specific business context (e.g. start date is before end date, price is within expected range).

It is always recommended to prevent attacks as early as possible in the processing of the user’s (could be attacker’s) request. Input validation can be used to detect unauthorised input before it is processed by the application.

Regular expressions (regex) are used to validate data entry and catch any invalid data input.

These are some basic rules for input validation:

It is also worth considering validation on relating parts of your system, even if not directly populated from an external source.

Input validation can be implemented using any programming technique as long as it allows enforcement of syntactic and semantic correctness. Some examples:

It is a common mistake to use black list validation in order to try to detect possibly dangerous characters and patterns like the apostrophe ' character, the string 1=1, or the <script> tag, but this is a flawed approach as it is trivial for an attacker to avoid getting caught by such filters*. Filters like that can also prevent authorised input like an apostrophe in a name, for example.

White list validation should be used for all input fields provided by the user. It involves defining exactly what is authorised, which means everything else is not authorised. For structured input fields (e.g. postcode, social security number, date), you can build very strong validation patterns using regular expressions.

Many web applications do not treat email addresses correctly due to common misconceptions about what constitutes a valid address.

A valid email address is case sensitive in the local portion of the address (left of the rightmost @ character).

A valid email address can have non-alphanumeric characters in the local-part (including + and @).

Following current best practice for validating an email address would be to*:

As the local-part of email addresses is case sensitive, it is important to store and compare email addresses correctly. To normalise an email address input, you would convert the domain part only to lowercase.

It seems most email providers do not actually recognise email addresses as unique if they have the same alphabetic letters, differing only by which letters are capitals (e.g. Susanne@ and suSanne@). That is why you can use capital letters randomly in your email address and it would still work.

Susanne Lundkvist

Related posts:

Moving apartments can be an overwhelming and time-consuming task, often causing stress. To simplify this process, we recommend hiring a professional moving company like “Three Movers.” We are…

The Flat design era is finally coming to an end… yup, and old trends are getting a makeover to renew themselves. One of them are the use of Gradients. 2018 it’s all about custom illustrations…

Mia has been collecting art for more than a century, from cultures around the world and across history. We now have more than 90,000 objects in our collection, and, like most museums, we have…