An Article from Aaron's Article ArchiveRuby Regexp Class Oddness
Photo: Desert FlowerYou are not logged in. Click here to log in.
Photo: Desert FlowerYou are not logged in. Click here to log in.
Here is one of my web log entries, perhaps from my Yakkity Yak page, What's New page, or one of my Astounding Adventures from my Geocaching section:
Ruby Regexp Class Oddness
Wednesday, 19 December 2007 9:52 AM MST
Yakkity Yak
This morning while working on an email log processing ruby script, I discovered this bit of strangeness with Ruby's Regexp regular expression class.
It appears the in-group match modifier (?i) (which should enable case insensitivity within the group) correctly modifies the A option, but does something strange on any subsequent options, apparently inverting the regular expression case so that string "B" doesn't match regular expression character "B".
Here's the same thing with a lowercase regex:
There is no odd inversion in this case. String character "B" doesn't match regex character "b" -- leading me to conclude that the (?i) modifier only applies within grouping's first option. This would be a perfectly fine expected behavior if when I use uppercase in the regex, the (?i) modifier didn't act on subsequent optional subsets and effectively invert them when they're uppercase.
It gets worse:
In the above, the first match of string "bb" with regex group option "Bb" makes it seem the (?i) modifier is acting correctly on both the first group option AND the second group option. However the failure of all other variations of capitalization of "bb" means there's something screwy.
My conclusion? Don't use (?ix-ix) modifiers (or at least (?i) anyway) inside groups with multiple options separated by the bar/pipe "|" character unless you stick the modifier in each and every such option, or unless you stick it outside the group with multiple options by enclosing the group in a non-capturing group with the modifier, thus:
Now THAT works as expected.
NOTE: I'm running ruby version 1.8.6 on Mac OS X Leopard. However I get the same results with 1.8.6 under FreeBSD too.
Update (29 April 2009):
I'm now running Ruby 1.9.1 on Mac OS X and it looks like things are fixed:
>> /((?i)A|B)/.match("a")
=> #<MatchData:0x613de8>
>> /((?i)A|B)/.match("A")
=> #<MatchData:0x608330>
>> /((?i)A|B)/.match("b")
=> #<MatchData:0x603024>
>> /((?i)A|B)/.match("B")
=> nil
It appears the in-group match modifier (?i) (which should enable case insensitivity within the group) correctly modifies the A option, but does something strange on any subsequent options, apparently inverting the regular expression case so that string "B" doesn't match regular expression character "B".
Here's the same thing with a lowercase regex:
>> /((?i)a|b)/.match("a")
=> #<MatchData:0x5fa2d0>
>> /((?i)a|b)/.match("A")
=> #<MatchData:0x5f5c58>
>> /((?i)a|b)/.match("b")
=> #<MatchData:0x5eeb9c>
>> /((?i)a|b)/.match("B")
=> nil
There is no odd inversion in this case. String character "B" doesn't match regex character "b" -- leading me to conclude that the (?i) modifier only applies within grouping's first option. This would be a perfectly fine expected behavior if when I use uppercase in the regex, the (?i) modifier didn't act on subsequent optional subsets and effectively invert them when they're uppercase.
It gets worse:
>> /((?i)Aa|Bb)/.match("aa")
=> #<MatchData:0x5d9eb8>
>> /((?i)Aa|Bb)/.match("aA")
=> #<MatchData:0x5d0980>
>> /((?i)Aa|Bb)/.match("Aa")
=> #<MatchData:0x5be370>
>> /((?i)Aa|Bb)/.match("AA")
=> #<MatchData:0x5b2ea8>
>> /((?i)Aa|Bb)/.match("bb")
=> #<MatchData:0x5a5870>
>> /((?i)Aa|Bb)/.match("bB")
=> nil
>> /((?i)Aa|Bb)/.match("Bb")
=> nil
>> /((?i)Aa|Bb)/.match("BB")
=> nil
In the above, the first match of string "bb" with regex group option "Bb" makes it seem the (?i) modifier is acting correctly on both the first group option AND the second group option. However the failure of all other variations of capitalization of "bb" means there's something screwy.
My conclusion? Don't use (?ix-ix) modifiers (or at least (?i) anyway) inside groups with multiple options separated by the bar/pipe "|" character unless you stick the modifier in each and every such option, or unless you stick it outside the group with multiple options by enclosing the group in a non-capturing group with the modifier, thus:
>> /(?i:(Aa|Bb))/.match("aa")
=> #<MatchData:0x58e8c8>
>> /(?i:(Aa|Bb))/.match("aA")
=> #<MatchData:0x589c10>
>> /(?i:(Aa|Bb))/.match("Aa")
=> #<MatchData:0x584f6c>
>> /(?i:(Aa|Bb))/.match("AA")
=> #<MatchData:0x5802c8>
>> /(?i:(Aa|Bb))/.match("bb")
=> #<MatchData:0x57b624>
>> /(?i:(Aa|Bb))/.match("bB")
=> #<MatchData:0x576980>
>> /(?i:(Aa|Bb))/.match("Bb")
=> #<MatchData:0x571cdc>
>> /(?i:(Aa|Bb))/.match("BB")
=> #<MatchData:0x56d038>
Now THAT works as expected.
NOTE: I'm running ruby version 1.8.6 on Mac OS X Leopard. However I get the same results with 1.8.6 under FreeBSD too.
Update (29 April 2009):
I'm now running Ruby 1.9.1 on Mac OS X and it looks like things are fixed:
irb(main):001:0> /((?i)A|B)/.match("a")
=> #<MatchData "a" 1:"a">
irb(main):002:0> /((?i)A|B)/.match("A")
=> #<MatchData "A" 1:"A">
irb(main):003:0> /((?i)A|B)/.match("b")
=> #<MatchData "b" 1:"b">
irb(main):004:0> /((?i)A|B)/.match("B")
=> #<MatchData "B" 1:"B">