8000 Merge branch 'master' into upstream · tricknotes/coderay@1d445c7 · GitHub
[go: up one dir, main page]
Skip to content

Commit 1d445c7

Browse files
kornykorny
committed
Merge branch 'master' into upstream
2 parents d9d6dd5 + 1e330f1 commit 1d445c7

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

98 files changed

+1536
-3133
lines changed

‎.gitignore

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,5 @@
11
.DS_Store
2+
.*~
23
coverage
34
pkg
45
spec/reports

‎.travis.yml< 6377 /code>

Lines changed: 8 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,15 +1,19 @@
11
rvm:
22
- 1.8.7
3-
- 1.9.2
3+
- ree
44
- 1.9.3
5+
- 2.0.0
6+
- ruby-head
57
- jruby-18mode
68
- jruby-19mode
9+
- jruby-head
710
- rbx-18mode
811
- rbx-19mode
9-
- ruby-head # test again later: RedCloth not compiling
10-
- jruby-head
11-
- ree
1212
branches:
1313
only:
1414
- master
15+
matrix:
16+
allow_failures:
17+
- rvm: rbx-18mode
18+
- rvm: rbx-19mode
1519
script: "rake test" # test:scanners"

‎Changes.textile

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,9 +4,26 @@ p=. _This files lists all changes in the CodeRay library since the 0.9.8 release
44

55
h2. Changes in 1.1
66

7+
* New scanner: Lua [#21, #22, thanks to Quintus]
8+
* New scanner: Sass [#93]
9+
* New scanner: Taskpaper [#39, thanks to shimomura]
710
* Diff scanner: Highlight inline changes in multi-line changes [#99]
11+
* JavaScript scanner: Highlight multi-line comments in diff correctly
12+
* Ruby scanner: Accept keywords as Ruby 1.9 hash keys [#126]
813
* Remove double-click toggle handler from HTML table output
14+
* Fixes to CSS scanner (floats, pseudoclasses)
15+
* CSS scanner uses @:id@ and @:tag@ now [#27]
16+
* Removed @Tokens#dump@, @Tokens.load@, @Tokens::Undumping@, and @zlib@ dependency. Nobody was using this, right?
17+
* Add .xaml file type [#121, thanks to Kozman Bálint]
18+
* @CodeRay::TokenKinds@ should not be frozen [#130, thanks to Gavin Kistner]
19+
* New token type @:id@ for CSS/Sass [#27]
20+
* New token type @:done@ for Taskpaper [#39]
21+
* New token type @:map@ for Lua, introducing a nice nested-shades trick [#22, thanks to Quintus and nathany]
922
* Display line numbers in HTML @:table@ mode even for single-line code (remove special case) [#41, thanks to Ariejan de Vroom]
23+
* Override Bootstrap's pre word-break setting for line numbers [#102, thanks to lightswitch05]
24+
* Fixed @:docstring@ token type style
25+
* @Plugin@ does not warn about fallback when default is defined
26+
* @Debug@ encoder refactored; use @DebugLint@ if you want strict checking now
1027

1128
h2. Changes in 1.0.9
1229

‎Gemfile

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -9,8 +9,8 @@ group :development do
99
gem "bundler", ">= 1.0.0"
1010
gem "rake"
1111
gem "RedCloth", RUBY_PLATFORM == 'java' ? ">= 4.2.7" : ">= 4.0.3"
12-
gem "term-ansicolor", '= 1.1.1'
13-
gem "shoulda-context", "~> 1.0.0"
12+
gem "term-ansicolor", '~> 1.2.2'
13+
gem "shoulda-context", "~> 1.1.2"
1414
gem "json" if RUBY_VERSION < '1.9'
1515
gem "rdoc"
1616
end

‎README.markdown

Lines changed: 6 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,8 @@
1-
# CodeRay [![Build Status](https://travis-ci.org/rubychan/coderay.png)](https://travis-ci.org/rubychan/coderay) [![Gem Version](https://badge.fury.io/rb/coderay.png)](http://badge.fury.io/rb/coderay)
1+
# CodeRay
2+
3+
[![Build Status](https://travis-ci.org/rubychan/coderay.png)](https://travis-ci.org/rubychan/coderay)
4+
[![Gem Version](https://badge.fury.io/rb/coderay.png)](http://badge.fury.io/rb/coderay)
5+
[![Dependency Status](https://gemnasium.com/rubychan/coderay.png)](https://gemnasium.com/rubychan/coderay)
26

37
## About
48

@@ -12,7 +16,7 @@ You put your code in, and you get it back colored; Keywords, strings, floats, co
1216

1317
### Dependencies
1418

15-
CodeRay needs Ruby 1.8.7+ or 1.9.2+. It also runs on Rubinius and JRuby.
19+
CodeRay needs Ruby 1.8.7, 1.9.3 or 2.0. It also runs on JRuby.
1620

1721
## Example Usage
1822

‎bench/bench.rb

Lines changed: 14 additions & 40 deletions
Original file line numberDiff line numberDiff line change
@@ -14,11 +14,10 @@
1414
lang = ARGV.fetch(0) do
1515
puts <<-HELP
1616
Usage:
17-
ruby bench.rb (c|ruby|dump) (null|text|tokens|count|statistic|yaml|html) [size in kB] [stream]
17+
ruby bench.rb (c|ruby) (null|text|tokens|count|statistic|yaml|html) [size in kB] [stream]
1818
1919
SIZE defaults to 100 kB (= 100,000 bytes).
2020
SIZE = 0 means the whole input.
21-
SIZE is ignored when dump is input.
2221
2322
-p generates a profile (slow! use with SIZE = 1)
2423
-o shows the output
@@ -48,10 +47,6 @@
4847
end
4948
end
5049

51-
$dump_input = lang == 'dump'
52-
$dump_output = format == 'dump'
53-
require 'coderay/helpers/gzip_simple.rb' if $dump_input
54-
5550
def here fn = nil
5651
return MYDIR unless fn
5752
File.join here, fn
@@ -66,59 +61,38 @@ def here fn = nil
6661

6762
data = nil
6863
File.open(here("#$filename." + lang), 'rb') { |f| data = f.read }
69-
if $dump_input
70-
@size = CodeRay::Tokens.load(data).text.size
71-
else
72-
raise 'Example file is empty.' if data.empty?
73-
unless @size.zero?
74-
data += data until data.size >= @size
75-
data = data[0, @size]
76-
end
77-
@size = data.size
64+
raise 'Example file is empty.' if data.empty?
65+
unless @size.zero?
66+
data += data until data.size >= @size
67+
data = data[0, @size]
7868
end
79-
69+
@size = data.size
70+
8071
options = {
8172
:tab_width => 2,
8273
# :line_numbers => :inline,
8374
:css => $style ? :style : :class,
8475
}
85-
$hl = CodeRay.encoder(format, options) unless $dump_output
76+
$hl = CodeRay.encoder(format, options)
8677
time = bm.report('CodeRay') do
8778
if $stream || true
88-
if $dump_input
89-
raise 'Can\'t stream dump.'
90-
elsif $dump_output
91-
raise 'Can\'t dump stream.'
92-
end
9379
$o = $hl.encode(data, lang, options)
9480
else
95-
if $dump_input
96-
tokens = CodeRay::Tokens.load data
97-
else
98-
tokens = CodeRay.scan(data, lang)
99-
end
81+
tokens = CodeRay.scan(data, lang)
10082
tokens.optimize! if $optimize
101-
if $dump_output
102-
$o = tokens.optimize.dump
103-
else
104-
$o = tokens.encode($hl)
105-
end
83+
$o = tokens.encode($hl)
10684
end
10785
end
108-
$file_created = here('test.' +
109-
($dump_output ? 'dump' : $hl.file_extension))
86+
$file_created = here('test.' + $hl.file_extension)
11087
File.open($file_created, 'wb') do |f|
11188
# f.write $o
11289
end
113-
Dir.chdir(here) do
114-
FileUtils.copy 'test.dump', 'example.dump' if $dump_output
115-
end
116-
90+
11791
time_real = time.real
118-
92+
11993
puts "\t%7.2f KB/s (%d.%d KB)" % [((@size / 1000.0) / time_real), @size / 1000, @size % 1000]
12094
puts $o if ARGV.include? '-o'
121-
95+
12296
end
12397
end
12498
puts "Files created: #$file_created"

‎bin/coderay

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -125,7 +125,7 @@ when 'highlight', nil
125125
end
126126

127127
if output_file
128-
output_format ||= CodeRay::FileType[output_file]
128+
output_format ||= CodeRay::FileType[output_file] || :plain
129129
else
130130
output_format ||= :terminal
131131
end
@@ -143,7 +143,6 @@ when 'highlight', nil
143143
if output_file
144144
File.open output_file, 'w'
145145
else
146-
$stdout.sync = true
147146
$stdout
148147
end
149148
CodeRay.encode(input, input_lang, output_format, :out => file)
@@ -156,8 +155,9 @@ when 'highlight', nil
156155
puts boom.message
157156
end
158157
# puts "I don't know this plugin: #{boom.message[/Could not load plugin (.*?): /, 1]}."
159-
rescue CodeRay::Scanners::Scanner::ScanError # FIXME: rescue Errno::EPIPE
160-
# this is sometimes raised by pagers; ignore [TODO: wtf?]
158+
rescue CodeRay::Scanners::Scanner::ScanError
159+
# this is sometimes raised by pagers; ignore
160+
# FIXME: rescue Errno::EPIPE
161161
ensure
162162
file.close if output_file
163163
end

‎lib/coderay.rb

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -166,7 +166,6 @@ class << self
166166
#
167167
# See also demo/demo_simple.
168168
def scan code, lang, options = {}, &block
169-
# FIXME: return a proxy for direct-stream encoding
170169
TokensProxy.new code, lang, options, block
171170
end
172171

‎lib/coderay/encoders/debug.rb

Lines changed: 5 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,6 @@ module Encoders
99
#
1010
# You cannot fully restore the tokens information from the
1111
# output, because consecutive :space tokens are merged.
12-
# Use Tokens#dump for caching purposes.
1312
#
1413
# See also: Scanners::Debug
1514
class Debug < Encoder
@@ -18,37 +17,26 @@ class Debug < Encoder
1817

1918
FILE_EXTENSION = 'raydebug'
2019

21-
def initialize options = {}
22-
super
23-
@opened = []
24-
end
25-
2620
def text_token text, kind
2721
if kind == :space
2822
@out << text
2923
else
30-
# TODO: Escape (
31-
text = text.gsub(/[)\\]/, '\\\\\0') if text.index(/[)\\]/)
32-
@out << kind.to_s << '(' << text << ')'
24+
text = text.gsub('\\', '\\\\\\\\') if text.index('\\')
25+
text = text.gsub(')', '\\\\)') if text.index(')')
26+
@out << "#{kind}(#{text})"
3327
end
3428
end
3529

3630
def begin_group kind
37-
@opened << kind
38-
@out << kind.to_s << '<'
31+
@out << "#{kind}<"
3932
end
4033

4134
def end_group kind
42-
if @opened.last != kind
43-
puts @out
44-
raise "we are inside #{@opened.inspect}, not #{kind}"
45-
end
46-
@opened.pop
4735
@out << '>'
4836
end
4937

5038
def begin_line kind
51-
@out << kind.to_s << '['
39+
@out << "#{kind}["
5240
end
5341

5442
def end_line kind

‎lib/coderay/encoders/debug_lint.rb

Lines changed: 55 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,55 @@
1+
module CodeRay
2+
module Encoders
3+
4+
# = Debug Lint Encoder
5+
#
6+
# Debug encoder with additional checks for:
7+
#
8+
# - empty tokens
9+
# - incorrect nesting
10+
#
11+
# It will raise an InvalidTokenStream exception when any of the above occurs.
12+
#
13+
# See also: Encoders::Debug
14+
class DebugLint < Debug
15+
16+
register_for :debug_lint
17+
18+
InvalidTokenStream = Class.new StandardError
19+
EmptyToken = Class.new InvalidTokenStream
20+
IncorrectTokenGroupNesting = Class.new InvalidTokenStream
21+
22+
def initialize options = {}
23+
super
24+
@opened = []
25+
end
26+
27+
def text_token text, kind
28+
raise EmptyToken, 'empty token' if text.empty?
29+
super
30+
end
31+
32+
def begin_group kind
33+
@opened << kind
34+
super
35+
end
36+
37+
def end_group kind
38+
raise IncorrectTokenGroupNesting, "We are inside #{@opened.inspect}, not #{kind} (end_group)" if @opened.pop != kind
39+
super
40+
end
41+
42+
def begin_line kind
43+
@opened << kind
44+
super
45+
end
46+
47+
def end_line kind
48+
raise IncorrectTokenGroupNesting, "We are inside #{@opened.inspect}, not #{kind} (end_line)" if @opened.pop != kind
49+
super
50+
end
51+
52+
end
53+
54+
end
55+
end

0 commit comments

Comments
 (0)
0