36 KiB
Regular expressions
Gutes Deutsche Lehrseite : http://www.regenechsen.de/phpwcms/index.php?regex_allg_einf
Sonntag, 16. Oktober 2016
16:36
| Zeichen | Bedeutung |
|---|---|
| . | Beliebiges Zeichen außer Newline. Im Modus DotAll (?s) wird auch Newline erkannt |
| ^ | Erkennt den Zeilenanfang |
| $ | Erkennt das Zeilenende |
| ...|...|... | Stellt Alternativen für das Suchmuster. Die erste auftretende Alternative im String wird gefunden. |
| (...) | Dient der Gruppierung von Suchmustern. Das gefundene Muster wird in ein Subpattern für spätere Verwendung gespeichert. |
| [...] | Zeichenklasse: die in den eckigen Klammern stehenden Zeichen werden als Alternative verwendet. Es können Bereiche angegeben werden: [a-p]. Einige Metazeichen haben hier andere Bedeutung. [^...] negiert die Klasse |
| \ | Backslash: hebt die besondere Bedeutung von Metazeichen auf: +?.*()^$[{|\ um diese literal suchen zu können |
| * | Erkennt vorhergehendes Element nicht oder mehrmals |
| + | Erkennt vorhergehendes Element ein- oder mehrmals |
| ? | Erkennt vorhergehendes Element nicht oder einmal ODER Hebt die Gierigkeit der anderen Quantifizierer auf (.*?), minimal nötige Menge wird gefunden ODER Leitet einen Modifikator ein (?i) ODER Leitet konditionale Regexe (?(....)....|....) sowie Assertions (?=...) ein |
| {x,y} | Erkennt vorhergehendes Element x- bis höchstens y-mal. ',y' ist optional: {x} erkennt das Element x-mal. {x,} erkennt das Element mindestens x-mal, aber auch beliebig häufiger. |
| \d | Erkennt Ziffern |
| \D | Erkennt Nicht-Ziffern |
| \w | Erkennt alphanumerische Zeichen und den Unterstrich |
| \W | Erkennt Nicht-\w-Zeichen |
| \s | Erkennt Whitespaces (Tab, Leerzeichen usw.) |
| \S | Erkennt Nicht-Whitespaces |
| \b | Erkennt Wortgrenzen |
| \B | Erkennt Nicht-Wortgrenzen |
| \Z | Erkennt das Ende des Strings bzw. das Zeichen unmittelbar vor dem Newline am Ende |
| \z | Erkennt das tatsächliche, physikalische Ende des Strings |
| \A | Erkennt den Anfang des Strings |
| \1...\9 | Rückbezüge. Sie verweisen auf ein zuvor in (...) gefundenes Subpattern. Bei entsprechend vielen Subpattern sind auch Zahlen größer 9 möglich |
| \nnn | Erkennt Oktalzahl nnn. Besonderheiten sind zu beachten |
| \xnn | Erkennt Hexadezimalzahl nn |
| (?:...) | Die Gruppierung wird nicht als Subpattern zwischengespeichert. |
| (?(Bedingungsmuster) Ja-Muster|Nein-Muster) | Konditionales Regex: erkennt das Ja-Muster, wenn das Bedingungsmuster in der Zeichenkette enthalten ist, ansonsten erkennt es das Nein-Muster. Das Nein-Muster ist optional. |
| …(?=…) | Positive Lookahead-Assertion. Das Muster vor der Klammer wird nur erkannt, wenn das Muster in der Klammer danach gefunden wird. Das Muster in der Klammer ist im Suchergebnis nicht enthalten. |
| …(?!…) | Negative Lookahead-Assertion. Das Muster vor der Klammer wird nur erkannt, wenn das Muster in der Klammer danach nicht gefunden wird. Das Muster in der Klammer ist im Suchergebnis nicht enthalten. |
| (?<=…)… | Positive Lookbehind-Assertion. Das Muster nach der Klammer wird nur erkannt, wenn das Muster in der Klammer davor gefunden wird. Das Muster in der Klammer ist im Suchergebnis nicht enthalten. |
| (?<!…)… | Negative Lookbehind-Assertion. Das Muster nach der Klammer wird nur erkannt, wenn das Muster in der Klammer davor nicht gefunden wird. Das Muster in der Klammer ist im Suchergebnis nicht enthalten. |
| (?Modifikator) | Schalter zum Setzen oder Ausschalten von Modifikationen zur Mustererkennung. Durch Aufführen eines Modifikators wird dieser eingeschaltet. Ein Minus-Zeichen davor schaltet ihn aus. Es dürfen mehrere Modifikatoren gleichzeitig eingetragen sein (?i-s).
|
In case you have the plugins installed, try Ctrl+R or in the TextFX -> TextFX Quick -> Find/Replace to get a sophisticated dialogue including a drop down for regular expressions and multi line search/replace.
This tutorial was based on an earlier, far more limited regular expression syntax. The examples are still the same at the date of writing, they require additions or upgrading to the new ways.
Notepad++ regular expressions use the standard PCRE (Perl) syntax, only departing from it in very minor ways. Complete documentation on the precise implementation is to be found on the implementer's website.
Another great tutorial is provided online at http://www.regular-expressions.info . More advanced material, specially about recursion, is to be found on the various pages of http://www.rexegg.com .
A french Sourceforge user, guy038, made a tutorial available in the French language. This is hosted at ici in a variety of formats.
In a regular expression (shortened into regex throughout), special characters interpreted are:
Single-character matches
., \c
Matches any character. If you check the box which says ". matches newline", the dot will indeed do that, enabling the "any" character to run over multiple lines. With the option unchecked, then . will only match characters within a line, and not the line ending characters (\r and \n)
\X
Matches a single non-combining characer followed by any number of combining characters. This is useful if you have a Unicode encoded text with accents as separate, combining characters.
\?
This allows you to use a character ? that would otherwise have a special meaning. For example, \[ would be interpreted as [ and not as the start of a character set. Adding the backslash (this is called escaping) works the other way round, as it makes special a character that otherwise isn't. For instance, \d stands for "a digit", while "d" is just an ordinary letter.
Non ASCII characters
\xnn
Specify a single chracter with code nn. What this stands for depends on the text encoding. For instance, \xE9 may match an é or a ? depending on the code page in an ANSI encoded document.
\x{nnnn}
Like above, but matches a full 16-bit Unicode character. If the document is ANSI encoded, this construct is invalid.
\Onnn
A single byte character whose code in octal is nnn.
\[.collating sequence.The character the collating sequence stands for. For instance, in Spanish, "ch" is a single letter, though it is written using two characters. That letter would be represented as .ch.. This trick also works with symbolic names of control characters, like .BEL. for the character of code 0x07. See also the discussion on character ranges.
Control characters
\a
The BEL control character 0x07 (alarm).
\b
The BS control character 0x08 (backspace). This is only allowed inside a character class definition. Otherwise, this means "a word boundary".
\e
The ESC control character 0x1B.
\f
The FF control character 0x0C (form feed).
\n
The LF control character 0x0A (line feed). This is the regular end of line under Unix systems.
\r
The CR control character 0x0D (carriage return). This is part of the DOS/Windows end of line sequence CR-LF, and was the EOL character on Mac 9 and earlier. OSX and later versions use \n.
\R
Any newline character.
\t
The TAB control character 0x09 (tab, or hard tab, horizontal tab).
\Ccharacter
The control character obtained from character by stripping all but its 6 lowest order bits. For instance, \C1, \CA and \Ca all stand for the SOH control character 0x01.
Ranges or kinds of characters
\...This indicates a set of characters, for example, [abc] means any of the characters a, b or c. You can also use ranges, for example [a-z] for any lower case character. You can use a collating sequence in character ranges, like in .ch.]-[.ll. (these are collating sequence in Spanish).
\^\...The complement of the characters in the set. For example, [^A-Za-z] means any character except an alphabetic character. Care should be taken with a complement list, as regular expressions are always multi-line, and hence [^ABC]* will match until the first A,B or C (or a, b or c if match case is off), including any newline characters. To confine the search to a single line, include the newline characters in the exception list, e.g. [^ABC\r\n].
\[:name:The whole character class named name. Most of the time, there is a single letter escape sequence for them - see below.
Recognised classes are:
alnum : ASCII letters and digits
alpha : ASCII letters
blank : spacing which is not a line terminator
cntrl : control characters
d , digit : decimal digits
graph : graphical character
l , lower : lowercase letters
print : printable characters
punct : punctuation characters: , " ' ? ! ; : # $ % & ( ) * + - / < > = @ [ ] \ ^ _ { } | ~
s , space : whitespace
u , upper : uppercase letters
unicode : any character with code point above 255
w , word : word character
xdigit : hexadecimal digits
\pshort name,\p{name}
Same as :name:. For instance, \pd and \p{digit} both stand for a digit, \d.
\Pshort name,\P{name]
Same as [^[:name:]] (not belonging to the class name).
Note that Unicode categories like in \p{Sc} or \p{Currency_Symbol}, they are flagged as an invalid regex in v6.6.6. This is because support would draw a large library in, which would have other uses.
\d
A digit in the 0-9 range, same as :digit:.
\D
Not a digit. Same as [^[:digit]].
\l
A lowercase letter. Same as [a-z] or :lower:.
NOTE: this will fall back on "a word character" if the "Match case" search option is off.
\L
Not a lower case letter. See note above.
\u
An uppercase letter. Same as :uper:. See note about lower case letters.
\U
Not an uppercase letter. Same note applies.
\w
A word character, which is a letter, digit or underscore. This appears not to depend on what the Scintilla component considers as word characters. Same as :word:.
\W
Not a word character. Same as :alnum: with the addition of the underscore.
\s
A spacing character: space, EOLs and tabs count. Same as :space:.
\S
Not a space.
\h
Horizontal spacing. This only matches space, tab and line feed.
\H
Not horizontal whitespace.
\v
Vertical whitespace. This encompasses the The VT, FF and CR control characters: 0x0B (vertical tab), 0x0D (carriage return) and 0x0C (form feed).
\V
Not vertical whitespace.
\[=primary key=All characters that differ from primary key by case, accent or similar alteration only. For example matches any of the characters: a, À, Á, Â, Ã, Ä, Å, A, à, á, â, ã, ä and å.
Multiplying operators
+
This matches 1 or more instances of the previous character, as many as it can. For example, Sa+m matches Sam, Saam, Saaam, and so on. [aeiou]+ matches consecutive strings of vowels.
*
This matches 0 or more instances of the previous character, as many as it can. For example, Sa*m matches Sm, Sam, Saam, and so on.
?
Zero or one of the last character. Thus Sa?m matches Sm and Sam, but not Saam.
*?
Zero or more of the previous group, but minimally: the shortest matching string, rather than the longest string as with the "greedy" * operator. Thus, m.*?o applied to the text margin-bottom: 0; will match margin-bo, whereas m.*o will match margin-botto.
+?
One or more of the previous group, but minimally.
{n}
Matches n copies of the element it applies to.
{n,}
Matches n' or more copies of the element it applies to.
{m,n}
Matches m to n copies of the element it applies to, as much it can.
{n,}?,{m,n}?
Like the above, but match as few copies as they can. Compare with *? and friends.
*+,?+,++,{n,}+,{m,n}+
These so called "possessive" variants of greedy repeat marks do not backtrack. This allows failures to be reported much earlier, which can boost performance significantly. But they will eliminate matches that would require backtracking to be found.
Example: matching ".*" against "abc"x will find "abc", because
" then abc"x then $ fails
" then abc" then x fails
" then abc then " succeeds.
However, matching "*+" against "abc"x will fail, because the possessive repeat factor prevented backtracking.
Anchors
Anchors match a position in the line, rather than a particular character.
^
This matches the start of a line (except when used inside a set, see above).
$
This matches the end of a line.
\<
This matches the start of a word using Scintilla's definitions of words.
\>
This matches the end of a word using Scintilla's definition of words.
\b
Matches either the start or end of a word.
\B
Not a word boundary.
\A, \'
The start of the matching string.
\z, \`
The end of the matching string.
\Z
Matches like \z with an optional sequence of newlines before it. This is equivalent to (?=\v*\z), which departs from the traditional Perl meaning for this escape.
Groups
(...)
<Parentheses mark a subset of the regular expression. The string matched by the contents of the parentheses ( ) can be re-used as a backreference or as part of a replace operation; see Substitutions, below.
Groups may be nested.
(?<some name>...), (?'some name'...),(?(some name)...)
Names this group some name.
\gn , \g{n}
The n-th subexpression, aka parenthesised group. Uing the second form has some small benefits, like n being more than 9, or disambiguating when n might be followed by digits. When n' is negative, groups are counted backwards, so that \g-2 is the second last matched group.
\g{something},\k<something>
The string matching the subexpression named something.
\digit
Backreference: \1 matches an additional occurence of a text matched by an earlier part of the regex. Example: This regular expression: ([Cc][Aa][Ss][Ee]).*\1 would match a line such as Case matches Case but not Case doesn't match cASE. A regex can have multiple subgroups, so \2, \3, etc can be used to match others (numbers advance left to right with the opening parenthesis of the group). So \n is a synonym for \gn, but doesn't support the extension syntax for the latter.
Readability enhancements
(:...)
A grouping construct that doesn't count as a subexpression, just grouping things for easier reading of the regex.
(?#...)
Comments. The whole group is for humans only and will be ignored in matching text.
Using the x flag modifier (see section below) is also a good way to improve readability in complex regular expressions.
Search modifiers
The following constructs control how matches condition other matches, or otherwise alter the way search is performed. For those readers familiar with Perl, \G is not supported.
\Q
Starts verbatim mode (Perl calls it "quoted"). In this mode, all characters are treated as-is, the only exception being the \E end verbatim mode sequence.
\E
Ends verbatim mode. Ths, "\Q\*+\Ea+" matches "\*+aaaa".
(?:flags-not-flags ...), (?:flags-not-flags:...)
Applies flags and not-flags to search inside the parentheses. Such a construct may have flags and may have not-flags - if it has neither, it is just a non-marking group, which is just a readability enhancer. The following flags are known:
i : case insensitive (default: off)
m : ^ and $ match embedded newlines (default: as per ". matches newline")
s: dot matches newline (default: as per ". matches newline")
x: Ignore unescaped whitespace in regex (default: off)
(?|expression using the alternation | operator)
If an alternation expression has subexpressions in some of its alternatives, you may want the subexpression counter not to be altered by what is in the other branches of the alternation. This construct will just do that.
For example, you get the following subexpressioncounter values:
# before ---------------branch-reset----------- after
/ ( a ) (?| x ( y ) z | (p (q) r) | (t) u (v) ) ( z ) /x
# 1 2 2 3 2 3 4
Without the construct, (p(q)r) would be group #3, and (t) group #5. With the constuct, they both report as group #2.
Control flow
Normally, a regular expression parses from left to right linerly. But you may need to change this behaviour.
|
The alternation operator, which allows matching either of a number of options, like in : one|two|three to match either of "one", "two" or "three". Matches are attempted from left to right. Use (?:) to match an empty string in such a construct.
(?n), (?signed-n)
Refers to subexpression #n. When a sign is present, go to the signed-n-th expression.
(?0), (?R)
Backtrack to start of pattern.
(?&name)
Backtrack to subexpression named name.
(?assertionyes-pattern|no-pattern)
Mathes yes-pattern if assertion is true, and no-pattern otherwise if provided. Supported assertions are:
(?=assert) (positive lookahead)
(?!assert) (negative lookahead)
(?(R)) (true if inside a recursion)
(?(Rn) (true if in a recursion to subexpression numbered n
PCRE doesn't treat recursion expressions like Perl does:
In PCRE (like Python, but unlike Perl), a recursive subpattern call is
always treated as an atomic group. That is, once it has matched some of
the subject string, it is never re-entered, even if it contains untried
alternatives and there is a subsequent matching failure.
\K
Resets matched text at this point. For instance, matching "foo\Kbar" will not match bar". It will match "foobar", but will pretend that only "bar" matches. Useful when you wish to replace only the tail of a matched subject and groups are clumsy to formulate.
Assertions
These special groups consume no characters. Their successful matching counts, but when they are done, matching starts over where it left.
(?=pattern)
If pattern matches, backtrack to start of pattern. This allows using logical AND for combining regexes.
For instance,
(?=.*:lower:)(?=.*:upper:).{6,}
tries finding a lowercase letter anywhere. On success it backtracks and searches for an uppercase letter. On yet another success, it checks whether the subject has at least 6 characters.
'"q(?=u)i" doesn't match "quit", because, as matching 'u' consumes 0 characters, matching "i" in the pattern fails at "u" i the subject.
(?!pattern)
Matches if pattern didn't match.
(?<=pattern)
Asserts that pattern matches before some token.
(?<pattern)
Asserts that pattern does not match before some token.
NOTE: pattern has to be of fixed length, so that the regex engine knows where to test the assertion.
(?>pattern)
Match pattern independently of surrounding patterns, and don't backtrack into it. Failure to match will cause the whole subject not to match.
Substitutions
\a,\e,\f,\n,\r,\t,\v
The corresponding control character, respectively BEL, ESC, FF, LF, CR, TAB and VT.
\Ccharacter", \xnn,\x{nnnn</i>}
Like in search patterns, respectively the control character with the same low order bits, the character with code 'nn and the character with code nnnn (requires Unicode encoding).
\l
Causes next character to output in lowercase
\L
Causes next characters to be output in lowercase, until a \E is found.
\u
Causes next character to output in uppercase
\U
Causes next characters to be output in uppercase, until a \E is found.
\E
Puts an end to forced case mode initiated by \L or \U.
$&, $MATCH, ${^MATCH}
The whole matched text.
$`, $PREMATCH, ${^PREMATCH}
The text between the previous and current match, or the text before the match if this is the first one.
$", $POSTMATCH, ${$POSTMATCH}
Everything that follows current match.
$LAST_SUBMATCH_RESULT, $^N
Returns what the last matching subexpression matched.
$+, $LAST_PAREN_MATCH
Returns what matched the last subexpression in the pattern.
$$
Returns $.
$n, ${n}, \n
Returns what matched the subexpression numbered n. Negative indices are not alowed.
$+{name}
Returns what matched subexpression named name.
Beispiele für Reguläre Ausdrücke
Im abschließenden Kapitel möchte ich verschiedene Beispiele für reguläre Ausdrücke geben, die zum einen im täglichen Gebrauch hilfreich sein können und zum anderen verschiedene in diesem Tutorial dargestellte Syntaxregeln verdeutlichen.
Die unten aufgeführten Regexe wurden verschiedenen Quellen entnommen.
Prüfung auf korrekte E-Mailadressen-Syntax
\\w-
Dieses Regex trifft auf 99% aller gängigen Mailadressen zu. Eine perfekte Prüfung auf Mailadressen ist nahezu unmöglich: der Aufwand und die Fehleranfälligkeit des Regexes steigt überproportional mit der Komplexität des Regexes.
{width="0.28125in" height="6.25e-2in"}
Prüfung auf erlaubte IP-Adresse
(25[0-5]|2[0-4][0-9]|[0-1]{1}[0-9]{2}|[1-9]{1}[0-9]{1}|[1-9])
\.(25[0-5]|2[0-4][0-9]|[0-1]{1}[0-9]{2}|[1-9]{1}[0-9]{1}|[1-9]
|0)\.(25[0-5]|2[0-4][0-9]|[0-1]{1}[0-9]{2}|[1-9]{1}[0-9]{1}|
1-9{1}|[0-9])
[Anmerkung: Das Regex wurde aus Gründen der Lesbarkeit auf mehrere Zeilen umgebrochen. Es muss natürlich ohne Zeilenumbruch verwendet werden.]
Die Form dieses Regexes sollte aus einer der Aufgaben bekannt sein: Prüfung auf ein (mehr oder weniger) gültiges Datum. Auch dort wurde das Regex in Teilprobleme zerlegt. Im Falle der IP-Adressen sind bestimmte Nummern ausgeschlossen (0.0.0.0 oder jeder Wert größer 255) und die Schreibweise ist individuell (192.0.0.1 oder 192.000.000.001): beides wird berücksichtigt.
{width="0.28125in" height="6.25e-2in"}
Prüfung auf mehrfach auftretende gleichen Empfänger in E-Maildresse
((.*?)@.*?,\s*)(\2@.*?(,\s*)?){2,}
Diese Prüfung geht von einer Auflistung von Emailadressen der Art
"person@web.de, person@gmx.de, person@weissnicht.de"
aus. Die Zeichen vor dem @-Zeichen werden gespeichert und im Subpattern 2 abgelegt ((.*?)@.*?,\s*). Der zweite Teil (\2@.*?(,\s*)?) verwendet das Ergebnis des Matches und prüft die nachfolgenden Adressen über einen Rückbezug auf das zweite Pattern \2. In Fällen, in denen die Adressen aber wie folgt geschrieben werden
"Person 1 <person@web.de>, Person 1 <person@gmx.de>, Person 1 <person@weissnicht.de>"
reicht dieses Regex nicht aus. Stattdessen sollte dann das universellere
(((.*?)\s*<)?(.*?)@.*?>?,\s*)(((.*?)\s*<)?\4@.*?>?(,\s*)?){2,}
verwendet werden.
[Achtung:]{.underline} in diesem Fall musste der Rückbezug geändert werden. Der Empfänger in der Mailadresse ist nunmehr im vierten Klammerpaar und wird daher mit \4 angesprochen. In beiden Fällen gibt die Ziffer in der geschweiften Klammer am Ende des Ausdrucks an, wie häufig der gleiche Empfänger zusätzlich aufgeführt werden muss.
{width="0.28125in" height="6.25e-2in"}
Prüfung auf mehrfach auftretende gleiche Domain in E-Mailadresse
(.*?(@.*?),\s*)(.*?\2(,\s*)?){3,}
Viel häufiger als die Mehrfachnennung des Empfängernamens kommt die Mehrfachnennung der Domain bei Spam vor:
Albert.a@web.de, berta.b@web.de, charlie.c@web.de, dora.d@web.de
Diese werden durch das obige Regex gefunden. Auch hier wird wieder ein Backreference verwendet.
Bedingte Prüfung auf eine Vielzahl von Ziffern im Betreff
Die Überschrift an sich ist schon kryptisch ;-) Gemeint ist, dass nur Betreffs gefunden werden, in denen eine Vielzahl von Ziffern vorkommen. Klassischer Fall, der in den selektiven Downloadfiltern verwendet wird. Allerdings birgt sowas eine Gefahr, denn zum Beispiel erhalten eBay-Käufer oder auch -Verkäufer Mails mit der 10-stelligen Artikelnummer und diese würden dann auch im Spam-Filter landen. Also muss ein Regex her, dass eine Bedingung in sich trägt, nämlich "finde nur Ziffern, wenn davor nicht 'Artikelnummer' steht".
.*?(?<!Artikelnummer:) \d{5,}
Hierbei wird eine negative Lookbehind-Assertion verwendet, die jede mindestens 5-stellige Ziffernfolge findet, die das Wort "Artikelnummer:" nicht davor hat. Sobald "Artikelnummer:" den 5 Ziffern vorausgeht, findet das Regex diese nicht mehr: in einem selektiven Downloadfilter verwendet bliebe diese Mail also nicht am Server hängen.
Prüfung auf mehrere aufeinander folgende Konsonanten
Verschiedene Spammer verwenden wechselnde Mailadressen von verschiedensten Domänen. Merkmal dieser Mailadressen ist, dass sehr viele Konsonaten aufeinander folgen: fndghklxrrstU@beispiel.com
Wenn man nicht generell die Domäne im selektiven Download sperren will, so muss man ein anderes Vorgehen wählen. Eine Methode ist es, eine Zeichengruppe zu definieren, die alles erlaubt, nur keine Vokale:
(?i)[^aeiou]{5,}.*@
Nimmt man mal an, dass in einem mehr oder weniger sinnvollen Namen nicht mehr als 5 Konsonanten direkt aufeinander folgen, so kann das obige Regex schon zum Filtern verwendet werden.
Anchors
Character Classes
POSIX
Assertions
|
Quantifiers
Add a ? to a quantifier to make it ungreedy. Escape Sequences
"Escaping" is a way of treating characters which have a special meaning in regular expressions literally, rather than as special characters. Common Metacharacters
The escape character is usually \ Special Characters
|
Groups and Ranges
Ranges are inclusive. Pattern Modifiers
* PCRE modifier String Replacement
Some regex implementations use \ instead of $. |
|---|