eWPTXv3 Notes Part5

I take notes while I study for the eWPTXv3. I separated all the notes according to INE’s six main topics.

Filter Evasion & WAF Bypass Techniques

Data Encoding

Data encoding refers to the process of converting information into a specific format that enables it to be efficiently transmitted, stored, and interpreted by both machines and humans.

This transformation ensures that diverse forms of data (such as text, images, multimedia, or files) can be accurately rendered and communicated across various platforms and technologies used on the internet.

Encoding typically involves translating raw data into standardized formats, such as Base64, URL encoding, or HTML entities, to maintain data integrity and compatibility with digital systems.

Encoding takes on an additional and critical role in penetration testing. Testers often explore how a web application processes and handles encoded input to identify potential weaknesses. For example, encoding is frequently used to craft payloads that include special characters or binary sequences, helping security professionals test the robustness of input validation mechanisms.

Manipulating data through encoding can also help in bypassing security filters, discovering hidden application behavior, or exploiting vulnerabilities. For instance, attackers may encode malicious inputs to evade detection mechanisms, making it easier to carry out attacks such as SQL Injection, Cross-Site Scripting (XSS), or other injection-based attacks.

Understanding how a web application interprets encoded data provides valuable insights into its internal logic and security posture. By systematically submitting various encoded payloads and analyzing the application’s response, penetration testers can uncover subtle flaws that might otherwise go unnoticed thereby strengthening the overall security of the system.

Character Sets (charsets)

A “charset,” short for character set, is a collection of characters, symbols, and glyphs that are associated with unique numeric codes or code points.

Character sets define how textual data is mapped to binary values in computing systems.

They play a fundamental role in encoding and decoding text, allowing computers to store, process, and display human-readable characters from different writing systems.

ASCII

ASCII → American Standard Code for Information Interchange.

It’s a widely used character encoding standard containing 128 characters to represent text and control characters in computers and communication equipment.

ASCII defines a set of codes to represent letters, numbers, punctuation, and control characters used in the English language and basic communication.

ASCII includes a total of 128 characters, each represented by a unique 7-bit binary code. These characters include uppercase and lowercase letters, digits, punctuation marks, and some control characters.

Limitations → ASCII is primarily designed for English text and doesn’t support characters from other languages or special symbols.

ASCII is a subset of many other character encodings, including more comprehensive standards like Unicode. The first 128 characters of the Unicode standard correspond to the ASCII characters.

Unicode

Recommended site → https://www.charset.org

Unicode is a character set standard that aims to encompass characters from all writing systems and languages used worldwide.

It enables computers to handle text and characters from diverse languages and scripts, making it essential for internationalization and multilingual communication.

“UTF” stands for “Unicode Transformation Format.” It refers to different character encoding schemes within the Unicode standard that are used to represent Unicode characters as binary data.

Character encoding is the representation in bytes of the symbols of a charset.

Unicode has three main character encoding schemes: UTF-8, UTF-16 and UTF-32.

UTF-8

UTF-8 is a variable-length character encoding scheme.

It uses 8-bit units (bytes) to represent characters.

ASCII characters are represented using a single byte. Non-ASCII characters are represented using multiple bytes, with the number of bytes varying based on the character’s code point.

UTF-8 is widely used on the web and in many applications due to its efficiency and compatibility with ASCII.

UTF-16

UTF-16 is a variable-length character encoding scheme.

It uses 16-bit units (two bytes) to represent characters.

Characters with code points below 65536 (BMP — Basic Multilingual Plane) are represented using two bytes.

Characters with higher code points (outside the BMP) are represented using four bytes (surrogate pairs).

UTF-16 is commonly used in programming languages like Java and Windows systems.

Why Encoding Is Important

Encoding important in website because if you dont specify the encoding type in the code and you use a different language than English (for example Greek, Arabic etc.) then your browser will not display the text correctly in clients browser.

You should specify encoding type in your code’s head tag with meta tag.

<meta charset=”utf-8"> 

HTML Encoding

HTML encoding, also referred to as HTML entity encoding, is the process of converting special characters and reserved symbols into a specific format known as HTML entities. This ensures that these characters are displayed correctly in web browsers and are not mistakenly interpreted as part of the HTML code itself.

HTML Encoding is essential for preserving the integrity of web content and plays a vital role in securing web applications particularly in mitigating threats like Cross-Site Scripting (XSS) attacks. By encoding potentially dangerous characters (such as <, >, or &), developers can prevent user-supplied input from being executed as malicious code in the browser.

HTML entities are predefined sequences that represent specific characters or symbols within HTML. They begin with an ampersand (&) and conclude with a semicolon (;). For example, the less-than symbol (<) is represented as < and the ampersand (&) as &.

When a browser encounters one of these entities in the HTML source, it interprets and renders the corresponding symbol visually, rather than treating it as part of the page’s structure or logic. This allows developers to safely include reserved characters in their content without disrupting the layout or behavior of the webpage.

For example &lt; is <

URL Encoding

URL encoding, also known as percent-encoding, is a method used to convert special, reserved, or non-ASCII characters into a standardized format that is safe for inclusion in URLs (Uniform Resource Locators) and URIs (Uniform Resource Identifiers).

Since URLs are used to locate and access resources on the internet, and some characters have specific meanings within a URL structure, encoding helps eliminate ambiguity and ensures proper interpretation by web browsers and servers.

This encoding technique replaces characters that are not safe for URLs with a percent sign (%) followed by two hexadecimal digits representing the ASCII code of the character. For example, a space character is encoded as %20. This conversion enables reliable data transmission across the web, as only a limited subset of characters is allowed within URLs.

URLs must consist only of characters from the US-ASCII character set. If a URL contains characters outside this allowed range — such as spaces, symbols, or non-English characters — they must be encoded to maintain compatibility and prevent parsing errors.

There are two main character categories relevant to URL encoding:

• Unreserved characters, which do not require encoding and include: [a–z], [A–Z], [0–9], -, ., _, and ~.
• Reserved characters, which have special roles in URLs and may require encoding depending on context. These include: :, /, ?, #, [, ], @, !, $, &, ', (, ), *, +, ,, ;, and =.

When you interact with a website your browser automatically applies URL encoding to any characters that need it. On the receiving end, the server typically decodes these characters to reconstruct the original input.

URL encoding is not a security mechanism. While it affects how data is represented and transmitted, it does not inherently protect against attacks. However, it can influence the attack surface by altering how payloads are interpreted, potentially helping or hindering certain exploits.

Recommended resource → https://www.w3schools.com/tags/ref_urlencode.ASP

URL encoding is only used in transmission of information within the URL. It has nothing to do with protecting the web application against injection attacks.

URL encoding is not sanitizing input, it’s just ensures the data reach to destination in the correct format.

Base64 Encoding

Base64 encoding is a technique used to convert binary data — such as images, audio files, or other non-text formats — into a textual representation that can be safely handled in environments that only support text. This is especially useful in scenarios like embedding data in email messages, transmitting files via web forms, or including binary content within URLs or JSON objects.

The core idea behind Base64 encoding is to transform binary data into a sequence of ASCII characters, ensuring compatibility with text-based systems that may not support raw binary input.

Base64 encoding uses a set of 64 different characters. These characters consist of letters (uppercase and lowercase), digits, and two additional characters (often + and /).

The encoding process works by taking three bytes (24 bits) of binary data at a time and dividing them into four 6-bit segments. Each of these segments is then mapped to a corresponding character from the Base64 character set. This is often referred to as 3-to-4 mapping, because three bytes of binary input produce four characters of Base64 output.

However, binary data isn’t always a perfect multiple of three bytes. To ensure the encoded output remains properly structured, padding is added when necessary. The padding character, usually the equals sign (=), is appended to the end of the encoded string to make the final output a multiple of four characters in length.

Filtering

Filtering in web application security refers to the practice of inspecting and controlling data input and output in a web application to prevent security vulnerabilities and protect against various types of attacks.

Filtering is a critical aspect of security because it helps ensure that data entering and leaving the application is safe, valid, and free from malicious content or potential exploits.

Input Filtering

This involves validating and sanitizing data received by the web application from users or external sources.

Input Filtering Techniques

Some common techniques for input filtering include data validation, input validation, and input sanitization.

Data validation checks whether the incoming data conforms to expected formats and constraints. Invalid or unexpected data should be rejected or sanitized.

Input validation goes a step further by not only checking data formats but also assessing data for potential security threats. It detects and rejects input that could be used for attacks, such as SQL injection payloads or malicious scripts.

Input sanitization involves cleaning or escaping input data to remove or neutralize potentially dangerous characters or content. For example, converting special characters to their HTML entities can prevent XSS attacks by rendering malicious scripts harmless.

Content Security Policy (CSP) is a security feature that controls which sources of content are allowed to be loaded by a web page. It helps prevent XSS attacks by specifying which domains are permitted sources for scripts, styles, images, and other resources.

Cross-Site Request Forgery (CSRF) Protection: Filtering mechanisms can be used to implement CSRF protection, ensuring that incoming requests have valid anti-CSRF tokens to prevent attackers from tricking users into performing actions they didn’t intend.

WAFs are security appliances or services that filter incoming HTTP requests to a web application. They use predefined rules and heuristics to detect and block malicious traffic.

Regular Expression Filtering: Regular expressions (regex) can be used to filter and validate data against complex patterns. However, improper regex usage can introduce security vulnerabilities, so careful crafting and testing of regex patterns are necessary.

Recommended site for regex filters to test against payloads→ https://regex101.com

How to approach a black-box perspective and automate the process of bypassing→ Use burp intruder with SecLists Fuzzing payloads. Attack type sniper with the payloads and make sure payload encoding is enabled. In settings choose follow redirects and start the attack. In the results we’re looking for difference in Length to identify.

searchsploit important tool for common exploits about common services.

Web Application Security and Defense Mechanisms

Web application security mechanisms are safeguards and measures put in place to protect web applications from a wide range of security threats and vulnerabilities.

These mechanisms are essential for ensuring the confidentiality, integrity, and availability of web applications and their associated data.

These mechanisms: Authentication, Authorization, Input Validation/Filtering, Session Management, CSRF Protection, Security Headers, Rate Limiting

Web application defense mechanisms are proactive tools and techniques designed to protect and defend web applications against various security threats and vulnerabilities.

These mechanisms: WAFs, IDS, IPS, Proxies

Evasion

Evasion in web application security testing refers to the practice of using various techniques and methods to bypass or circumvent security mechanisms and controls put in place to protect a web application.

The primary goal of evasion techniques is to deceive or trick security measures in order to deliver malicious payloads or exploit vulnerabilities within the target application.

Evasion Techniques

Bypassing WAFs/Proxy Rules: Evasion techniques may involve encoding, obfuscation, or fragmentation of malicious payloads to bypass the WAF’s detection rules.

Evading Intrusion Detection Systems (IDS): Evasion techniques can be used to hide or modify the payload of an attack so that it goes undetected by the IDS.

Circumventing Input Validation Filters: Evasion may involve crafting input that seems legitimate but can still exploit vulnerabilities in the application.

Avoiding Rate Limiting and Authentication Controls: Attackers may use evasion techniques to avoid triggering rate-limiting mechanisms or to bypass authentication and authorization controls.

Squid Proxy

Squid is a widely used open-source proxy server and web cache daemon.

It primarily operates as a proxy server, which means it acts as an intermediary between client devices (such as computers or smartphones) and web servers, facilitating requests and responses between them.

Obfuscation

Main resource of obfuscation notes are from → https://github.com/0xInfection/Awesome-WAF?tab=readme-ov-file#evasion-techniques highly recommended resource

Recommended resource → https://portswigger.net/web-security/essential-skills/obfuscating-attacks-using-encodings

Obfuscation refers to the deliberate act of making code, payloads, scripts, binaries, or network traffic difficult to understand or analyze, with the goal of evading detection, delaying analysis, or bypassing defensive mechanisms.

Method:

Encoding payload to different encodings.

You can encode whole payload, or some parts of it and test recursively.

Techniques:

Case Toggling

Some poorly developed WAFs filter selectively specific case WAFs.

We can combine upper and lower case characters for developing efficient payloads.

Example: <ScRipT>alert()</sCRipT>

URL Encoding

Encode normal payloads with % encoding/URL encoding.

Example: %3CsvG%2Fx%3D%22%3E%22%2FoNloaD%3Dconfirm%28%29%2F%2F

Unicode Normalization

ASCII characters in unicode encoding provide great variants for bypassing.

You can encode entire/part of the payload for obtaining results.

Example: <marquee onstart=\\u0070r\\u06f\\u006dpt()>

HTML Representation

Often web apps encode special characters into HTML encoding and render them accordingly.

This leads us to basic bypass cases with HTML encoding (numeric/generic).

Example: "><img src=x onerror=confirm()> (General form)

Example: "><img src=x onerror=confirm()> (Numeric reference)

Mixed Encoding

Sometimes, WAF rules often tend to filter out a specific type of encoding.

This type of filters can be bypassed by mixed encoding payloads.

Example:

<A HREF="h
tt  p://6   6.000146.0x7.147/">XSS</A>

Using Comments

Comments obfuscate standard payload vectors.

Example: <!--><script>alert/**/()/**/</script>

Double Encoding

Often WAF filters tend to encode characters to prevent attacks.

However poorly developed filters (no recursion filters) can be bypassed with double encoding.

Example: %253Cscript%253Ealert()%253C%252Fscript%253E

Wildcard Obfuscation

Recommended resource → https://medium.com/@themiddleblue/waf-evasion-techniques-718026d693d8

Globbing patterns are used by various command-line utilities to work with multiple files.

We can tweak them to execute system commands.

Specific to remote code execution vulnerabilities on linux systems.

Example: /???/??t /???/??ss??

Dynamic Payload Generation

Different programming languages have different syntaxes and patterns for concatenation.

This allows us to effectively generate payloads that can bypass many filters and rules.

Example: <script>eval('al'+'er'+'t()')</script>

Junk Characters

Normal payloads get filtered out easily.

Adding some junk chars helps avoid detection (specific cases only).

They often help in confusing regex based firewalls.

Example: <script>+-+-1-+-+alert(1)</script>

Hex Escaping

Another option when injecting into a string context is to use hex escapes, which represent characters using their hexadecimal code point, prefixed with \x. For example, the lowercase letter a is represented by \x61.

Just like unicode escapes, these will be decoded client-side as long as the input is evaluated as a string.

Example:eval("\x61lert")

Note that you can sometimes also obfuscate SQL statements in a similar manner using the prefix 0x. For example, 0x53454c454354 may be decoded to form the SELECT keyword.

Octal Escaping

Octal escaping works in pretty much the same way as hex escaping, except that the character references use a base-8 numbering system rather than base-16. These are prefixed with a standalone backslash, meaning that the lowercase letter a is represented by \141.

Example:eval("\141lert(1)")