8000 New Ruby 2.1, 2.2, 2.3 Syntax by korny · Pull Request #189 · rubychan/coderay · GitHub
[go: up one dir, main page]

Skip to content

New Ruby 2.1, 2.2, 2.3 Syntax #189

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 10 commits into from
Feb 13, 2016
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions Changes.textile
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,10 @@ p=. _This files lists all changes in the CodeRay library since the 0.9.8 release
h2. Changes in 1.1.1

* SQL scanner: fix open strings [#163, thanks to Adam]
* Ruby scanner: Accept number literal suffixes @r@ and @i@ (Ruby 2.1)
* Ruby scanner: Accept quoted hash keys like @{ "a": boss }@ (Ruby 2.2)
* Ruby scanner: Accept save navigation operator @&.@ (Ruby 2.3)
* Ruby scanner: Accept squiggly heredoc @<<~@ (Ruby 2.3)

h2. Changes in 1.1

Expand Down
19 changes: 13 additions & 6 deletions lib/coderay/scanners/ruby.rb
Original file line number Diff line number Diff line change
Expand Up @@ -164,15 +164,19 @@ def scan_tokens encoder, options
end

elsif match = scan(/ ' (?:(?>[^'\\]*) ')? | " (?:(?>[^"\\\#]*) ")? /mx)
encoder.begin_group :string
if match.size == 1
kind = check(self.class::StringState.simple_key_pattern(match)) ? :key : :string
encoder.begin_group kind
encoder.text_token match, :delimiter
state = self.class::StringState.new :string, match == '"', match # important for streaming
state = self.class::StringState.new kind, match == '"', match # important for streaming
else
kind = value_expected == true && scan(/:/) ? :key : :string
encoder.begin_group kind
encoder.text_token match[0,1], :delimiter
encoder.text_token match[1..-2], :content if match.size > 2
encoder.text_token match[-1,1], :delimiter
encoder.end_group :string
encoder.end_group kind
encoder.text_token ':', :operator if kind == :key
value_expected = false
end

Expand All @@ -191,11 +195,14 @@ def scan_tokens encoder, options
encoder.text_token match, :error
method_call_expected = false
else
encoder.text_token match, self[1] ? :float : :integer # TODO: send :hex/:octal/:binary
kind = self[1] ? :float : :integer # TODO: send :hex/:octal/:binary
match << 'r' if match !~ /e/i && scan(/r/)
match << 'i' if scan(/i/)
encoder.text_token match, kind
end
value_expected = false

elsif match = scan(/ [-+!~^\/]=? | [:;] | [*|&]{1,2}=? | >>? /x)
elsif match = scan(/ [-+!~^\/]=? | [:;] | &\. | [*|&]{1,2}=? | >>? /x)
value_expected = true
encoder.text_token match, :operator

Expand All @@ -208,7 +215,7 @@ def scan_tokens encoder, options
encoder.end_group kind
heredocs ||= [] # create heredocs if empty
heredocs << self.class::StringState.new(kind, quote != "'", delim,
self[1] == '-' ? :indented : :linestart)
self[1] ? :indented : :linestart)
value_expected = false

elsif value_expected && match = scan(/#{patterns::FANCY_STRING_START}/o)
Expand Down
2 changes: 1 addition & 1 deletion lib/coderay/scanners/ruby/patterns.rb
Original file line number Diff line number Diff line change
Expand Up @@ -114,7 +114,7 @@ module Ruby::Patterns # :nodoc: all
# NOTE: This is not completely correct, but
# nobody needs heredoc delimiters ending with \n.
HEREDOC_OPEN = /
<< (-)? # $1 = float
<< ([-~])? # $1 = float
(?:
( [A-Za-z_0-9]+ ) # $2 = delim
|
Expand Down
8 changes: 8 additions & 0 deletions lib/coderay/scanners/ruby/string_state.rb
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,14 @@ class StringState < Struct.new :type, :interpreted, :delim, :heredoc,
end
end

def self.simple_key_pattern delim
if delim == "'"
/ (?> (?: [^\\']+ | \\. )* ) ' : /mx
else
/ (?> (?: [^\\"\#]+ | \\. | \#\$[\\"] | \#\{[^\{\}]+\} | \#(?!\{) )* ) " : /mx
end
end

def initialize kind, interpreted, delim, heredoc = false
if heredoc
pattern = heredoc_pattern delim, interpreted, heredoc == :indented
Expand Down
0