Use Jekyll.configuration to build a full configuration for use w/Jekyll.
Whatever: foo.bar
" in "_site/index.html" + And I should see "Second document's output:Use Jekyll.configuration to build a full configuration for use w/Jekyll.
Whatever: foo.bar
" in "_site/index.html" + + Scenario: Documents have an output attribute, which is the converted HTML based on site.config + Given I have an "index.html" page that contains "Second document's output: {{ site.documents[2].output }}" + And I have fixture collections + And I have a "_config.yml" file with content: + """ + kramdown: + guess_lang: false + collections: + - methods + """ + And I'm using kramdown v2 + When I run jekyll build + Then I should get a zero exit status + Then the _site directory should exist + And I should see "Second document's output:Use Jekyll.configuration to build a full configuration for use w/Jekyll.
Whatever: foo.bar
" in "_site/index.html" Scenario: Filter documents by where Given I have an "index.html" page that contains "{% assign items = site.methods | where: 'whatever','foo.bar' %}Item count: {{ items.size }}" diff --git a/features/create_sites.feature b/features/create_sites.feature index c1f759316a7..7352e5eefa6 100644 --- a/features/create_sites.feature +++ b/features/create_sites.feature @@ -175,17 +175,17 @@ Feature: Create sites Given I have a _posts directory And I have the following post: | title | date | layout | content | - | entry1 | 2020-12-31 | post | content for entry1. | + | entry1 | 2040-12-31 | post | content for entry1. | | entry2 | 2007-12-31 | post | content for entry2. | When I run jekyll build Then I should get a zero exit status And the _site directory should exist And I should see "content for entry2" in "_site/2007/12/31/entry2.html" - And the "_site/2020/12/31/entry1.html" file should not exist + And the "_site/2040/12/31/entry1.html" file should not exist When I run jekyll build --future Then I should get a zero exit status And the _site directory should exist - And the "_site/2020/12/31/entry1.html" file should exist + And the "_site/2040/12/31/entry1.html" file should exist Scenario: Basic site with layouts, posts and related posts Given I have a _layouts directory diff --git a/features/include_relative_tag.feature b/features/include_relative_tag.feature new file mode 100644 index 00000000000..9cbdcf122e7 --- /dev/null +++ b/features/include_relative_tag.feature @@ -0,0 +1,60 @@ +Feature: include_relative Tag + In order to share content across several closely related pages + As a hacker who likes to blog + I want to be able to include snippets in my site's pages and documents relative to current file + + Scenario: Include a file relative to a post + Given I have a _posts directory + And I have a _posts/snippets directory + And I have the following post: + | title | date | content | + | Star Wars | 2018-09-02 | {% include_relative snippets/welcome_para.md %} | + And I have an "_posts/snippets/welcome_para.md" file that contains "Welcome back Dear Reader!" + When I run jekyll build + Then I should get a zero exit status + And the _site directory should exist + And I should see "Welcome back Dear Reader!" in "_site/2018/09/02/star-wars.html" + + Scenario: Include a nested file relative to a post + Given I have a _posts directory + And I have a _posts/snippets directory + And I have a _posts/snippets/welcome_para directory + And I have the following post: + | title | date | content | + | Star Wars | 2018-09-02 | {% include_relative snippets/welcome_para.md %} | + And I have an "_posts/snippets/welcome_para.md" file that contains "{% include_relative snippets/welcome_para/greeting.md %} Dear Reader!" + And I have an "_posts/snippets/welcome_para/greeting.md" file that contains "Welcome back" + When I run jekyll build + Then I should get a zero exit status + And the _site directory should exist + And I should see "Welcome back Dear Reader!" in "_site/2018/09/02/star-wars.html" + + Scenario: Include a nested file relative to a post as an excerpt + Given I have a _posts directory + And I have a _posts/snippets directory + And I have a _posts/snippets/welcome_para directory + And I have a "_posts/2018-09-02-star-wars.md" file with content: + """ + {% include_relative snippets/welcome_para.md %} + + Hello World + """ + And I have an "_posts/snippets/welcome_para.md" file that contains "{% include_relative snippets/welcome_para/greeting.md %} Dear Reader!" + And I have an "_posts/snippets/welcome_para/greeting.md" file that contains "Welcome back" + And I have an "index.md" page that contains "{% for post in site.posts %}{{ post.excerpt }}{% endfor %}" + When I run jekyll build + Then I should get a zero exit status + And the _site directory should exist + And I should see "Welcome back Dear Reader!" in "_site/2018/09/02/star-wars.html" + And I should see "Welcome back Dear Reader!" in "_site/index.html" + + Scenario: Include a nested file relative to a page at root + Given I have a snippets directory + And I have a snippets/welcome_para directory + And I have a "index.md" page that contains "{% include_relative snippets/welcome_para.md %}" + And I have a "snippets/welcome_para.md" file that contains "{% include_relative snippets/welcome_para/greeting.md %} Dear Reader!" + And I have a "snippets/welcome_para/greeting.md" file that contains "Welcome back" + When I run jekyll build + Then I should get a zero exit status + And the _site directory should exist + And I should see "Welcome back Dear Reader!" in "_site/index.html" diff --git a/features/incremental_rebuild.feature b/features/incremental_rebuild.feature index 2ced57a6d81..2d22d2e1940 100644 --- a/features/incremental_rebuild.feature +++ b/features/incremental_rebuild.feature @@ -67,6 +67,25 @@ Feature: Incremental rebuild And the _site directory should exist And I should see "Basic Site with include tag: Regenerated by Jekyll" in "_site/index.html" + Scenario: Rebuild when a dependency of document in custom collection_dir is changed + Given I have a _includes directory + And I have a configuration file with "collections_dir" set to "collections" + And I have a collections/_posts directory + And I have the following post within the "collections" directory: + | title | date | layout | content | + | Wargames | 2009-03-27 | default | Basic Site with include tag: {% include about.html %} | + And I have an "_includes/about.html" file that contains "Generated by Jekyll" + When I run jekyll build -I + Then I should get a zero exit status + And the _site directory should exist + And I should see "Basic Site with include tag: Generated by Jekyll" in "_site/2009/03/27/wargames.html" + When I wait 1 second + Then I have an "_includes/about.html" file that contains "Regenerated by Jekyll" + When I run jekyll build -I + Then I should get a zero exit status + And the _site directory should exist + And I should see "Basic Site with include tag: Regenerated by Jekyll" in "_site/2009/03/27/wargames.html" + Scenario: A themed-site and incremental regeneration Given I have a configuration file with "theme" set to "test-theme" And I have an "index.md" page that contains "Themed site" diff --git a/features/rendering.feature b/features/rendering.feature index 4c4e28f6cf5..0b42d44ba27 100644 --- a/features/rendering.feature +++ b/features/rendering.feature @@ -5,6 +5,19 @@ Feature: Rendering But I want to make it as simply as possible So render with Liquid and place in Layouts + Scenario: Rendering a site with parentheses in its path name + Given I have a blank site in "omega(beta)" + And I have an "omega(beta)/test.md" page with layout "simple" that contains "Hello World" + And I have an omega(beta)/_includes directory + And I have an "omega(beta)/_includes/head.html" file that contains "Snippet" + And I have a configuration file with "source" set to "omega(beta)" + And I have an omega(beta)/_layouts directory + And I have an "omega(beta)/_layouts/simple.html" file that contains "{% include head.html %}: {{ content }}" + When I run jekyll build --profile + Then I should get a zero exit status + And I should see "Snippet:Hello World
" in "_site/test.html" + And I should see "_layouts/simple.html" in the build output + Scenario: When receiving bad Liquid Given I have a "index.html" page with layout "simple" that contains "{% include invalid.html %}" And I have a simple layout that contains "{{ content }}" diff --git a/features/site_configuration.feature b/features/site_configuration.feature index 6dc6f5b09fa..362bc0dab3e 100644 --- a/features/site_configuration.feature +++ b/features/site_configuration.feature @@ -203,6 +203,28 @@ Feature: Site configuration And I should see "Post Layout:content for entry1.
\n built at 2013-04-09T09:22:00-10:00" in "_site/2013/04/09/entry1.html" And I should see "Post Layout:content for entry2.
\n built at 2013-04-09T13:14:00-10:00" in "_site/2013/04/09/entry2.html" + Scenario: Generate proper dates with explicitly set timezone (using non-half hour offset ) + Given I have a _layouts directory + And I have a page layout that contains "Page Layout: {{ site.posts.size }}" + And I have a post layout that contains "Post Layout: {{ content }} built at {{ page.date | date_to_xmlschema }}" + And I have an "index.html" page with layout "page" that contains "site index page" + And I have a configuration file with: + | key | value | + | timezone | Australia/Eucla | + And I have a _posts directory + And I have the following posts: + | title | date | layout | content | + | entry1 | 2013-04-09 23:22 +0400 | post | content for entry1. | + | entry2 | 2013-04-10 03:14 +0400 | post | content for entry2. | + When I run jekyll build + Then I should get a zero exit status + And the _site directory should exist + And I should see "Page Layout: 2" in "_site/index.html" + And the "_site/2013/04/10/entry1.html" file should exist + And the "_site/2013/04/10/entry2.html" file should exist + And I should see "Post Layout:content for entry1.
\n built at 2013-04-10T04:07:00\+08:45" in "_site/2013/04/10/entry1.html" + And I should see "Post Layout:content for entry2.
\n built at 2013-04-10T07:59:00\+08:45" in "_site/2013/04/10/entry2.html" + Scenario: Limit the number of posts generated by most recent date Given I have a _posts directory And I have a configuration file with: diff --git a/features/step_definitions.rb b/features/step_definitions.rb index fd05c68f406..0913a271c67 100644 --- a/features/step_definitions.rb +++ b/features/step_definitions.rb @@ -185,6 +185,12 @@ # +Given("I'm using kramdown v{int}") do |int| + skip_this_scenario unless Kramdown::VERSION.to_i == int.to_i +end + +# + Given(%r!^I wait (\d+) second(s?)$!) do |time, _| sleep(time.to_f) end diff --git a/features/theme.feature b/features/theme.feature index 650c167577e..3941abc8d86 100644 --- a/features/theme.feature +++ b/features/theme.feature @@ -57,6 +57,17 @@ Feature: Writing themes And I should see "From your site." in "_site/assets/application.coffee" And I should see "From your site." in "_site/assets/base.js" + Scenario: A theme with *just* layouts + Given I have a configuration file with "theme" set to "test-theme-skinny" + And I have an "index.html" page with layout "home" that contains "The quick brown fox." + When I run jekyll build + Then I should get a zero exit status + And the _site directory should exist + And I should see "Message: The quick brown fox." in "_site/index.html" + But I should not see "_includes" in the build output + And I should not see "_sass" in the build output + And I should not see "assets" in the build output + Scenario: Requiring dependencies of a theme Given I have a configuration file with "theme" set to "test-dependency-theme" When I run jekyll build diff --git a/jekyll.gemspec b/jekyll.gemspec index e00cdb001ab..9a226e2160a 100644 --- a/jekyll.gemspec +++ b/jekyll.gemspec @@ -33,14 +33,18 @@ Gem::Specification.new do |s| s.add_runtime_dependency("addressable", "~> 2.4") s.add_runtime_dependency("colorator", "~> 1.0") s.add_runtime_dependency("em-websocket", "~> 0.5") - s.add_runtime_dependency("i18n", "~> 0.7") + s.add_runtime_dependency("i18n", ">= 0.7", "< 2") s.add_runtime_dependency("jekyll-sass-converter", "~> 1.0") s.add_runtime_dependency("jekyll-watch", "~> 2.0") - s.add_runtime_dependency("kramdown", "~> 1.14") s.add_runtime_dependency("liquid", "~> 4.0") s.add_runtime_dependency("mercenary", "~> 0.3.3") s.add_runtime_dependency("pathutil", "~> 0.9") rouge_versions = ENV["ROUGE_VERSION"] ? ["~> #{ENV["ROUGE_VERSION"]}"] : [">= 1.7", "< 4"] s.add_runtime_dependency("rouge", *rouge_versions) s.add_runtime_dependency("safe_yaml", "~> 1.0") + + # Depend on kramdown. For kramdown v2, extra gems are required. + # https://kramdown.gettalong.org/news.html#kramdown-200-released + kramdown_versions = ENV["KRAMDOWN_VERSION"] ? ["~> #{ENV["KRAMDOWN_VERSION"]}"] : [">= 1.17", "< 3"] + s.add_runtime_dependency("kramdown", *kramdown_versions) end diff --git a/lib/jekyll/commands/new.rb b/lib/jekyll/commands/new.rb index b123015641e..97549b16d53 100644 --- a/lib/jekyll/commands/new.rb +++ b/lib/jekyll/commands/new.rb @@ -87,12 +87,23 @@ def gemfile_contents gem "jekyll-feed", "~> 0.6" end -# Windows does not include zoneinfo files, so bundle the tzinfo-data gem -gem "tzinfo-data", platforms: [:mingw, :mswin, :x64_mingw, :jruby] +# Windows and JRuby does not include zoneinfo files, so bundle the tzinfo-data gem +# and associated library. +platforms :mingw, :x64_mingw, :mswin, :jruby do + gem "tzinfo", ">= 1", "< 3" + gem "tzinfo-data" +end # Performance-booster for watching directories on Windows -gem "wdm", "~> 0.1.0" if Gem.win_platform? +gem "wdm", "~> 0.1.0", :install_if => Gem.win_platform? + +# kramdown v2 ships without the gfm parser by default. If you're using +# kramdown v1, comment out this line. +gem "kramdown-parser-gfm" +# Lock `http_parser.rb` gem to `v0.6.x` on JRuby builds since newer versions of the gem +# do not have a Java counterpart. +gem "http_parser.rb", "~> 0.6.0", :platforms => [:jruby] RUBY end diff --git a/lib/jekyll/commands/serve.rb b/lib/jekyll/commands/serve.rb index 7efeb72d496..bffa648edd1 100644 --- a/lib/jekyll/commands/serve.rb +++ b/lib/jekyll/commands/serve.rb @@ -202,6 +202,7 @@ def webrick_opts(opts) :JekyllOptions => opts, :DoNotReverseLookup => true, :MimeTypes => mime_types, + :MimeTypesCharset => mime_types_charset, :DocumentRoot => opts["destination"], :StartCallback => start_callback(opts["detach"]), :StopCallback => stop_callback(opts["detach"]), @@ -379,6 +380,11 @@ def mime_types end private + + def mime_types_charset + SafeYAML.load_file(File.expand_path("serve/mime_types_charset.json", __dir__)) + end + def read_file(source_dir, file_path) File.read(Jekyll.sanitized_path(source_dir, file_path)) end diff --git a/lib/jekyll/commands/serve/mime_types_charset.json b/lib/jekyll/commands/serve/mime_types_charset.json new file mode 100644 index 00000000000..017469b9ccd --- /dev/null +++ b/lib/jekyll/commands/serve/mime_types_charset.json @@ -0,0 +1,71 @@ +{ + "application/javascript": "UTF-8", + "application/json": "UTF-8", + "application/manifest+json": "UTF-8", + "application/vnd.syncml+xml": "UTF-8", + "application/vnd.syncml.dm+wbxml": "UTF-8", + "application/vnd.syncml.dm+xml": "UTF-8", + "application/vnd.syncml.dmddf+xml": "UTF-8", + "application/vnd.wap.wbxml": "UTF-8", + "text/cache-manifest": "UTF-8", + "text/calendar": "UTF-8", + "text/coffeescript": "UTF-8", + "text/css": "UTF-8", + "text/csv": "UTF-8", + "text/html": "UTF-8", + "text/jade": "UTF-8", + "text/jsx": "UTF-8", + "text/less": "UTF-8", + "text/markdown": "UTF-8", + "text/mathml": "UTF-8", + "text/mdx": "UTF-8", + "text/n3": "UTF-8", + "text/plain": "UTF-8", + "text/prs.lines.tag": "UTF-8", + "text/richtext": "UTF-8", + "text/sgml": "UTF-8", + "text/shex": "UTF-8", + "text/slim": "UTF-8", + "text/spdx": "UTF-8", + "text/stylus": "UTF-8", + "text/tab-separated-values": "UTF-8", + "text/troff": "UTF-8", + "text/turtle": "UTF-8", + "text/uri-list": "UTF-8", + "text/vcard": "UTF-8", + "text/vnd.curl": "UTF-8", + "text/vnd.curl.dcurl": "UTF-8", + "text/vnd.curl.mcurl": "UTF-8", + "text/vnd.curl.scurl": "UTF-8", + "text/vnd.familysearch.gedcom": "UTF-8", + "text/vnd.fly": "UTF-8", + "text/vnd.fmi.flexstor": "UTF-8", + "text/vnd.graphviz": "UTF-8", + "text/vnd.in3d.3dml": "UTF-8", + "text/vnd.in3d.spot": "UTF-8", + "text/vnd.sun.j2me.app-descriptor": "UTF-8", + "text/vnd.wap.wml": "UTF-8", + "text/vnd.wap.wmlscript": "UTF-8", + "text/vtt": "UTF-8", + "text/x-asm": "UTF-8", + "text/x-c": "UTF-8", + "text/x-component": "UTF-8", + "text/x-fortran": "UTF-8", + "text/x-handlebars-template": "UTF-8", + "text/x-java-source": "UTF-8", + "text/x-lua": "UTF-8", + "text/x-markdown": "UTF-8", + "text/x-nfo": "UTF-8", + "text/x-opml": "UTF-8", + "text/x-pascal": "UTF-8", + "text/x-processing": "UTF-8", + "text/x-sass": "UTF-8", + "text/x-scss": "UTF-8", + "text/x-setext": "UTF-8", + "text/x-sfv": "UTF-8", + "text/x-suse-ymp": "UTF-8", + "text/x-uuencode": "UTF-8", + "text/x-vcalendar": "UTF-8", + "text/x-vcard": "UTF-8", + "text/yaml": "UTF-8" +} diff --git a/lib/jekyll/commands/serve/servlet.rb b/lib/jekyll/commands/serve/servlet.rb index 567ec313cbe..f3fcf339c1c 100644 --- a/lib/jekyll/commands/serve/servlet.rb +++ b/lib/jekyll/commands/serve/servlet.rb @@ -137,6 +137,7 @@ class Servlet < WEBrick::HTTPServlet::FileHandler def initialize(server, root, callbacks) # So we can access them easily. @jekyll_opts = server.config[:JekyllOptions] + @mime_types_charset = server.config[:MimeTypesCharset] set_defaults super end @@ -172,27 +173,28 @@ def do_GET(req, res) end end - validate_and_ensure_charset(req, res) + conditionally_inject_charset(res) res.header.merge!(@headers) rtn end # rubocop:enable Naming/MethodName - # - private - def validate_and_ensure_charset(_req, res) - key = res.header.keys.grep(%r!content-type!i).first - typ = res.header[key] - unless typ =~ %r!;\s*charset=! - res.header[key] = "#{typ}; charset=#{@jekyll_opts["encoding"]}" - end + # + # Inject charset based on Jekyll config only if our mime-types database contains + # the charset metadata. + # + # Refer `script/vendor-mimes` in the repository for further details. + def conditionally_inject_charset(res) + typ = res.header["content-type"] + return unless @mime_types_charset.key?(typ) + return if %r!;\s*charset=!.match?(typ) + + res.header["content-type"] = "#{typ}; charset=#{@jekyll_opts["encoding"]}" end # - - private def set_defaults hash_ = @jekyll_opts.fetch("webrick", {}).fetch("headers", {}) DEFAULTS.each_with_object(@headers = hash_) do |(key, val), hash| diff --git a/lib/jekyll/configuration.rb b/lib/jekyll/configuration.rb index f64264c9362..27c6f5cd34f 100644 --- a/lib/jekyll/configuration.rb +++ b/lib/jekyll/configuration.rb @@ -81,6 +81,7 @@ class Configuration < Hash "smart_quotes" => "lsquo,rsquo,ldquo,rdquo", "input" => "GFM", "hard_wrap" => false, + "guess_lang" => true, "footnote_nr" => 1, "show_warnings" => false, }, diff --git a/lib/jekyll/converters/markdown/kramdown_parser.rb b/lib/jekyll/converters/markdown/kramdown_parser.rb index b7caab551d4..6ed3d7178ad 100644 --- a/lib/jekyll/converters/markdown/kramdown_parser.rb +++ b/lib/jekyll/converters/markdown/kramdown_parser.rb @@ -18,6 +18,7 @@ def initialize(config) @config = config["kramdown"] || {} @highlighter = nil setup + load_dependencies end # Setup and normalize the configuration: @@ -30,6 +31,8 @@ def initialize(config) def setup @config["syntax_highlighter"] ||= highlighter @config["syntax_highlighter_opts"] ||= {} + @config["syntax_highlighter_opts"]["default_lang"] ||= "plaintext" + @config["syntax_highlighter_opts"]["guess_lang"] = @config["guess_lang"] @config["coderay"] ||= {} # XXX: Legacy. modernize_coderay_config make_accessible @@ -47,6 +50,24 @@ def convert(content) end private + + def load_dependencies + return if Kramdown::VERSION.to_i < 2 + if @config["input"] == "GFM" + Jekyll::External.require_with_graceful_fail("kramdown-parser-gfm") + end + + if highlighter == "coderay" + Jekyll::External.require_with_graceful_fail("kramdown-syntax-coderay") + end + + # `mathjax` emgine is bundled within kramdown-2.x and will be handled by + # kramdown itself. + if (math_engine = @config["math_engine"]) && math_engine != "mathjax" + Jekyll::External.require_with_graceful_fail("kramdown-math-#{math_engine}") + end + end + def make_accessible(hash = @config) hash.keys.each do |key| hash[key.to_sym] = hash[key] diff --git a/lib/jekyll/convertible.rb b/lib/jekyll/convertible.rb index bfd03986329..8a851f7c466 100644 --- a/lib/jekyll/convertible.rb +++ b/lib/jekyll/convertible.rb @@ -39,7 +39,7 @@ def read_yaml(base, name, opts = {}) begin self.content = File.read(@path || site.in_source_dir(base, name), - Utils.merged_file_read_opts(site, opts)) + **Utils.merged_file_read_opts(site, opts)) if content =~ Document::YAML_FRONT_MATTER_REGEXP self.content = $POSTMATCH self.data = SafeYAML.load(Regexp.last_match(1)) diff --git a/lib/jekyll/document.rb b/lib/jekyll/document.rb index 061839f13b5..a3971998bab 100644 --- a/lib/jekyll/document.rb +++ b/lib/jekyll/document.rb @@ -266,7 +266,7 @@ def read(opts = {}) else begin merge_defaults - read_content(opts) + read_content(**opts) read_post_data rescue StandardError => e handle_read_error(e) @@ -445,8 +445,8 @@ def merge_defaults end private - def read_content(opts) - self.content = File.read(path, Utils.merged_file_read_opts(site, opts)) + def read_content(**opts) + self.content = File.read(path, **Utils.merged_file_read_opts(site, opts)) if content =~ YAML_FRONT_MATTER_REGEXP self.content = $POSTMATCH data_file = SafeYAML.load(Regexp.last_match(1)) diff --git a/lib/jekyll/drops/site_drop.rb b/lib/jekyll/drops/site_drop.rb index 1cb636d8825..996126829f9 100644 --- a/lib/jekyll/drops/site_drop.rb +++ b/lib/jekyll/drops/site_drop.rb @@ -8,8 +8,7 @@ class SiteDrop < Drop mutable false def_delegator :@obj, :site_data, :data - def_delegators :@obj, :time, :pages, :static_files, :documents, - :tags, :categories + def_delegators :@obj, :time, :pages, :static_files, :tags, :categories private def_delegator :@obj, :config, :fallback_data @@ -39,6 +38,16 @@ def collections @site_collections ||= @obj.collections.values.sort_by(&:label).map(&:to_liquid) end + # `Site#documents` cannot be memoized so that `Site#docs_to_write` can access the + # latest state of the attribute. + # + # Since this method will be called after `Site#pre_render` hook, + # the `Site#documents` array shouldn't thereafter change and can therefore be + # safely memoized to prevent additional computation of `Site#documents`. + def documents + @documents ||= @obj.documents + end + # `{{ site.related_posts }}` is how posts can get posts related to # them, either through LSI if it's enabled, or through the most # recent posts. diff --git a/lib/jekyll/entry_filter.rb b/lib/jekyll/entry_filter.rb index d016e0c2043..91d7961c94b 100644 --- a/lib/jekyll/entry_filter.rb +++ b/lib/jekyll/entry_filter.rb @@ -31,9 +31,12 @@ def relative_to_source(entry) def filter(entries) entries.reject do |e| - unless included?(e) - special?(e) || backup?(e) || excluded?(e) || symlink?(e) - end + # Reject this entry if it is a symlink. + next true if symlink?(e) + # Do not reject this entry if it is included. + next false if included?(e) + # Reject this entry if it is special, a backup file, or excluded. + special?(e) || backup?(e) || excluded?(e) end end diff --git a/lib/jekyll/excerpt.rb b/lib/jekyll/excerpt.rb index 1df15e09a53..82a43f3a85c 100644 --- a/lib/jekyll/excerpt.rb +++ b/lib/jekyll/excerpt.rb @@ -128,36 +128,47 @@ def render_with_liquid? # # Returns excerpt String - LIQUID_TAG_REGEX = %r!{%-?\s*(\w+).+\s*-?%}!m + LIQUID_TAG_REGEX = %r!{%-?\s*(\w+)\s*.*?-?%}!m MKDWN_LINK_REF_REGEX = %r!^ {0,3}\[[^\]]+\]:.+$! def extract_excerpt(doc_content) head, _, tail = doc_content.to_s.partition(doc.excerpt_separator) - # append appropriate closing tag (to a Liquid block), to the "head" if the - # partitioning resulted in leaving the closing tag somewhere in the "tail" - # partition. - if head.include?("{%") - head =~ LIQUID_TAG_REGEX - tag_name = Regexp.last_match(1) + # append appropriate closing tag(s) (for each Liquid block), to the `head` + # if the partitioning resulted in leaving the closing tag somewhere + # in the `tail` partition. - if liquid_block?(tag_name) && head.match(%r!{%-?\s*end#{tag_name}\s*-?%}!).nil? - print_build_warning + if head.include?("{%") + modified = false + tag_names = head.scan(LIQUID_TAG_REGEX) + tag_names.flatten! + tag_names.reverse_each do |tag_name| + next unless liquid_block?(tag_name) + next if head =~ endtag_regex_stash(tag_name) + + modified = true head << "\n{% end#{tag_name} %}" end + print_build_warning if modified end - if tail.empty? - head - else - head.to_s.dup << "\n\n" << tail.scan(MKDWN_LINK_REF_REGEX).join("\n") - end + return head if tail.empty? + + head << "\n\n" << tail.scan(MKDWN_LINK_REF_REGEX).join("\n") end private + def endtag_regex_stash(tag_name) + @endtag_regex_stash ||= {} + @endtag_regex_stash[tag_name] ||= %r!{%-?\s*end#{tag_name}.*?\s*-?%}!m + end + def liquid_block?(tag_name) - Liquid::Template.tags[tag_name].superclass == Liquid::Block + return false unless tag_name.is_a?(String) + return false unless Liquid::Template.tags[tag_name] + + Liquid::Template.tags[tag_name].ancestors.include?(Liquid::Block) rescue NoMethodError Jekyll.logger.error "Error:", "A Liquid tag in the excerpt of #{doc.relative_path} couldn't be " \ @@ -167,12 +178,13 @@ def liquid_block?(tag_name) def print_build_warning Jekyll.logger.warn "Warning:", "Excerpt modified in #{doc.relative_path}!" - Jekyll.logger.warn "", - "Found a Liquid block containing separator '#{doc.excerpt_separator}' and has " \ - "been modified with the appropriate closing tag." - Jekyll.logger.warn "", - "Feel free to define a custom excerpt or excerpt_separator in the document's " \ - "Front Matter if the generated excerpt is unsatisfactory." + Jekyll.logger.warn "", "Found a Liquid block containing the excerpt separator" \ + " #{doc.excerpt_separator.inspect}. " + Jekyll.logger.warn "", "The block has been modified with the appropriate" \ + " closing tag." + Jekyll.logger.warn "", "Feel free to define a custom excerpt or" \ + " excerpt_separator in the document's front matter" \ + " if the generated excerpt is unsatisfactory." end end end diff --git a/lib/jekyll/liquid_renderer.rb b/lib/jekyll/liquid_renderer.rb index 55e4e0fb014..39bc639ff9e 100644 --- a/lib/jekyll/liquid_renderer.rb +++ b/lib/jekyll/liquid_renderer.rb @@ -53,7 +53,9 @@ def self.format_error(error, path) private def filename_regex - @filename_regex ||= %r!\A(#{source_dir}/|#{theme_dir}/|\W*)(.*)!i + @filename_regex ||= begin + %r!\A(#{Regexp.escape(source_dir)}/|#{Regexp.escape(theme_dir.to_s)}/|/*)(.*)!i + end end def new_profile_hash diff --git a/lib/jekyll/mime.types b/lib/jekyll/mime.types index af68d359e11..10f713c6ed7 100644 --- a/lib/jekyll/mime.types +++ b/lib/jekyll/mime.types @@ -5,36 +5,47 @@ application/andrew-inset ez application/applixware aw application/atom+xml atom application/atomcat+xml atomcat +application/atomdeleted+xml atomdeleted application/atomsvc+xml atomsvc +application/atsc-dwd+xml dwd +application/atsc-held+xml held +application/atsc-rsat+xml rsat application/bdoc bdoc +application/calendar+xml xcs application/ccxml+xml ccxml +application/cdfx+xml cdfx application/cdmi-capability cdmia application/cdmi-container cdmic application/cdmi-domain cdmid application/cdmi-object cdmio application/cdmi-queue cdmiq +application/cpl+xml cpl application/cu-seeme cu application/dash+xml mpd +application/dash-patch+xml mpp application/davmount+xml davmount application/docbook+xml dbk application/dssc+der dssc application/dssc+xml xdssc -application/ecmascript ecma +application/ecmascript ecma es application/emma+xml emma +application/emotionml+xml emotionml application/epub+zip epub application/exi exi +application/express exp +application/fdt+xml fdt application/font-tdpfr pfr -application/font-woff woff -application/font-woff2 woff2 application/geo+json geojson application/gml+xml gml application/gpx+xml gpx application/gxf gxf application/gzip gz +application/hjson hjson application/hyperstudio stk application/inkml+xml ink inkml application/ipfix ipfix -application/java-archive jar war ear +application/its+xml its +application/java-archive ear jar war application/java-serialized-object ser application/java-vm class application/javascript js mjs @@ -42,6 +53,7 @@ application/json json m application/json5 json5 application/jsonml+json jsonml application/ld+json jsonld +application/lgr+xml lgr application/lost+xml lostxml application/mac-binhex40 hqx application/mac-compactpro cpt @@ -49,32 +61,40 @@ application/mads+xml mads application/manifest+json webmanifest application/marc mrc application/marcxml+xml mrcx -application/mathematica ma nb mb +application/mathematica ma mb nb application/mathml+xml mathml application/mbox mbox +application/media-policy-dataset+xml mpf application/mediaservercontrol+xml mscml application/metalink+xml metalink application/metalink4+xml meta4 application/mets+xml mets +application/mmt-aei+xml maei +application/mmt-usd+xml musd application/mods+xml mods application/mp21 m21 mp21 -application/mp4 mp4s m4p +application/mp4 m4p mp4s application/msword doc dot application/mxf mxf -application/octet-stream bin dms lrf mar so dist distz pkg bpk dump elc deploy exe dll deb dmg iso img msi msp msm buffer +application/n-quads nq +application/n-triples nt +application/node cjs +application/octet-stream bin bpk buffer deb deploy dist distz dll dmg dms dump elc exe img iso lrf mar msi msm msp pkg so application/oda oda application/oebps-package+xml opf application/ogg ogx application/omdoc+xml omdoc -application/onenote onetoc onetoc2 onetmp onepkg +application/onenote onepkg onetmp onetoc onetoc2 application/oxps oxps +application/p2p-overlay+xml relo application/patch-ops-error+xml xer application/pdf pdf application/pgp-encrypted pgp -application/pgp-signature asc sig +application/pgp-keys asc +application/pgp-signature sig application/pics-rules prf application/pkcs10 p10 -application/pkcs7-mime p7m p7c +application/pkcs7-mime p7c p7m application/pkcs7-signature p7s application/pkcs8 p8 application/pkix-attr-cert ac @@ -84,14 +104,19 @@ application/pkix-pkipath pkipat application/pkixcmp pki application/pls+xml pls application/postscript ai eps ps +application/provenance+xml provx application/prs.cww cww application/pskc+xml pskcxml -application/rdf+xml rdf +application/raml+yaml raml +application/rdf+xml owl rdf application/reginfo+xml rif application/relax-ng-compact-syntax rnc application/resource-lists+xml rl application/resource-lists-diff+xml rld application/rls-services+xml rs +application/route-apd+xml rapd +application/route-s-tsid+xml sls +application/route-usd+xml rusd application/rpki-ghostbusters gbr application/rpki-manifest mft application/rpki-roa roa @@ -104,9 +129,12 @@ application/scvp-cv-response scs application/scvp-vp-request spq application/scvp-vp-response spp application/sdp sdp +application/senml+xml senmlx +application/sensml+xml sensmlx application/set-payment-initiation setpay application/set-registration-initiation setreg application/shf+xml shf +application/sieve sieve siv application/smil+xml smi smil application/sparql-query rq application/sparql-results+xml srx @@ -115,9 +143,17 @@ application/srgs+xml grxml application/sru+xml sru application/ssdl+xml ssdl application/ssml+xml ssml +application/swid+xml swidtag application/tei+xml tei teicorpus application/thraud+xml tfi application/timestamped-data tsd +application/toml toml +application/trig trig +application/ttml+xml ttml +application/ubjson ubj +application/urc-ressheet+xml rsheet +application/urc-targetdesc+xml td +application/vnd.1000minds.decision-model+xml 1km application/vnd.3gpp.pic-bw-large plb application/vnd.3gpp.pic-bw-small psb application/vnd.3gpp.pic-bw-var pvb @@ -126,12 +162,13 @@ application/vnd.3m.post-it-notes pwn application/vnd.accpac.simply.aso aso application/vnd.accpac.simply.imp imp application/vnd.acucobol acu -application/vnd.acucorp atc acutc +application/vnd.acucorp acutc atc application/vnd.adobe.air-application-installer-package+zip air application/vnd.adobe.formscentral.fcdt fcdt application/vnd.adobe.fxp fxp fxpl application/vnd.adobe.xdp+xml xdp application/vnd.adobe.xfdf xfdf +application/vnd.age age application/vnd.ahead.space ahead application/vnd.airzip.filesecure.azf azf application/vnd.airzip.filesecure.azs azs @@ -143,20 +180,25 @@ application/vnd.anser-web-certificate-issue-initiation cii application/vnd.anser-web-funds-transfer-initiation fti application/vnd.antix.game-component atx application/vnd.apple.installer+xml mpkg +application/vnd.apple.keynote key application/vnd.apple.mpegurl m3u8 +application/vnd.apple.numbers numbers +application/vnd.apple.pages pages application/vnd.apple.pkpass pkpass application/vnd.aristanetworks.swi swi application/vnd.astraea-software.iota iota application/vnd.audiograph aep +application/vnd.balsamiq.bmml+xml bmml application/vnd.blueice.multipass mpm application/vnd.bmi bmi application/vnd.businessobjects rep application/vnd.chemdraw+xml cdxml application/vnd.chipnuts.karaoke-mmd mmd application/vnd.cinderella cdy +application/vnd.citationstyles.style+xml csl application/vnd.claymore cla application/vnd.cloanto.rp9 rp9 -application/vnd.clonk.c4group c4g c4d c4f c4p c4u +application/vnd.clonk.c4group c4d c4f c4g c4p c4u application/vnd.cluetrust.cartomobile-config c11amc application/vnd.cluetrust.cartomobile-config-pkg c11amz application/vnd.commonspace csp @@ -174,10 +216,11 @@ application/vnd.curl.car car application/vnd.curl.pcurl pcurl application/vnd.dart dart application/vnd.data-vision.rdz rdz -application/vnd.dece.data uvf uvvf uvd uvvd +application/vnd.dbf dbf +application/vnd.dece.data uvd uvf uvvd uvvf application/vnd.dece.ttml+xml uvt uvvt -application/vnd.dece.unspecified uvx uvvx -application/vnd.dece.zip uvz uvvz +application/vnd.dece.unspecified uvvx uvx +application/vnd.dece.zip uvvz uvz application/vnd.denovo.fcselayout-link fe_launch application/vnd.dna dna application/vnd.dolby.mlp mlp @@ -199,10 +242,10 @@ application/vnd.ezpix-album ez2 application/vnd.ezpix-package ez3 application/vnd.fdf fdf application/vnd.fdsn.mseed mseed -application/vnd.fdsn.seed seed dataless +application/vnd.fdsn.seed dataless seed application/vnd.flographit gph application/vnd.fluxtime.clip ftc -application/vnd.framemaker fm frame maker book +application/vnd.framemaker book fm frame maker application/vnd.frogans.fnc fnc application/vnd.frogans.ltf ltf application/vnd.fsc.weblaunch fsc @@ -248,7 +291,7 @@ application/vnd.hp-pcl pcl application/vnd.hp-pclxl pclxl application/vnd.hydrostatix.sof-data sfd-hdstx application/vnd.ibm.minipay mpy -application/vnd.ibm.modcap afp listafp list3820 +application/vnd.ibm.modcap afp list3820 listafp application/vnd.ibm.rights-management irm application/vnd.ibm.secure-container sc application/vnd.iccprofile icc icm @@ -268,7 +311,7 @@ application/vnd.jam jam application/vnd.jcp.javame.midlet-rms rms application/vnd.jisp jisp application/vnd.joost.joda-archive joda -application/vnd.kahootz ktz ktr +application/vnd.kahootz ktr ktz application/vnd.kde.karbon karbon application/vnd.kde.kchart chrt application/vnd.kde.kformula kfo @@ -280,7 +323,7 @@ application/vnd.kde.kword kwd kw application/vnd.kenameaapp htke application/vnd.kidspiration kia application/vnd.kinar kne knp -application/vnd.koan skp skd skt skm +application/vnd.koan skd skm skp skt application/vnd.kodak-descriptor sse application/vnd.las.las+xml lasxml application/vnd.llamagraphics.life-balance.desktop lbd @@ -293,6 +336,7 @@ application/vnd.lotus-organizer org application/vnd.lotus-screencam scm application/vnd.lotus-wordpro lwp application/vnd.macports.portpkg portpkg +application/vnd.mapbox-vector-tile mvt application/vnd.mcd mcd application/vnd.medcalcdata mc1 application/vnd.mediastation.cdkey cdkey @@ -313,7 +357,7 @@ application/vnd.mophun.certificate mpc application/vnd.mozilla.xul+xml xul application/vnd.ms-artgalry cil application/vnd.ms-cab-compressed cab -application/vnd.ms-excel xls xlm xla xlc xlt xlw +application/vnd.ms-excel xla xlc xlm xls xlt xlw application/vnd.ms-excel.addin.macroenabled.12 xlam application/vnd.ms-excel.sheet.binary.macroenabled.12 xlsb application/vnd.ms-excel.sheet.macroenabled.12 xlsm @@ -326,16 +370,16 @@ application/vnd.ms-officetheme thmx application/vnd.ms-outlook msg application/vnd.ms-pki.seccat cat application/vnd.ms-pki.stl stl -application/vnd.ms-powerpoint ppt pps pot +application/vnd.ms-powerpoint pot pps ppt application/vnd.ms-powerpoint.addin.macroenabled.12 ppam application/vnd.ms-powerpoint.presentation.macroenabled.12 pptm application/vnd.ms-powerpoint.slide.macroenabled.12 sldm application/vnd.ms-powerpoint.slideshow.macroenabled.12 ppsm application/vnd.ms-powerpoint.template.macroenabled.12 potm -application/vnd.ms-project mpp mpt +application/vnd.ms-project mpt application/vnd.ms-word.document.macroenabled.12 docm application/vnd.ms-word.template.macroenabled.12 dotm -application/vnd.ms-works wps wks wcm wdb +application/vnd.ms-works wcm wdb wks wps application/vnd.ms-wpl wpl application/vnd.ms-xpsdocument xps application/vnd.mseq mseq @@ -343,7 +387,7 @@ application/vnd.musician mus application/vnd.muvee.style msty application/vnd.mynfc taglet application/vnd.neurolanguage.nlu nlu -application/vnd.nitf ntf nitf +application/vnd.nitf nitf ntf application/vnd.noblenet-directory nnd application/vnd.noblenet-sealer nns application/vnd.noblenet-web nnw @@ -373,7 +417,9 @@ application/vnd.oasis.opendocument.text-template ott application/vnd.oasis.opendocument.text-web oth application/vnd.olpc-sugar xo application/vnd.oma.dd2+xml dd2 +application/vnd.openblox.game+xml obgx application/vnd.openofficeorg.extension oxt +application/vnd.openstreetmap.data+xml osm application/vnd.openxmlformats-officedocument.presentationml.presentation pptx application/vnd.openxmlformats-officedocument.presentationml.slide sldx application/vnd.openxmlformats-officedocument.presentationml.slideshow ppsx @@ -385,7 +431,7 @@ application/vnd.openxmlformats-officedocument.wordprocessingml.template dotx application/vnd.osgeo.mapguide.package mgp application/vnd.osgi.dp dp application/vnd.osgi.subsystem esa -application/vnd.palm pdb pqa oprc +application/vnd.palm oprc pdb pqa application/vnd.pawaafile paw application/vnd.pg.format str application/vnd.pg.osasli ei6 @@ -397,7 +443,8 @@ application/vnd.previewsystems.box box application/vnd.proteus.magazine mgz application/vnd.publishare-delta-tree qps application/vnd.pvi.ptid1 ptid -application/vnd.quark.quarkxpress qxd qxt qwd qwt qxl qxb +application/vnd.quark.quarkxpress qwd qwt qxb qxd qxl qxt +application/vnd.rar rar application/vnd.realvnc.bed bed application/vnd.recordare.musicxml mxl application/vnd.recordare.musicxml+xml musicxml @@ -418,7 +465,8 @@ application/vnd.shana.informed.package ipk application/vnd.simtech-mindmapper twd twds application/vnd.smaf mmf application/vnd.smart.teacher teacher -application/vnd.solent.sdkm+xml sdkm sdkd +application/vnd.software602.filler.form+xml fo +application/vnd.solent.sdkm+xml sdkd sdkm application/vnd.spotfire.dxp dxp application/vnd.spotfire.sfs sfs application/vnd.stardivision.calc sdc @@ -446,8 +494,9 @@ application/vnd.symbian.install sis si application/vnd.syncml+xml xsm application/vnd.syncml.dm+wbxml bdm application/vnd.syncml.dm+xml xdm +application/vnd.syncml.dmddf+xml ddf application/vnd.tao.intent-module-archive tao -application/vnd.tcpdump.pcap pcap cap dmp +application/vnd.tcpdump.pcap cap dmp pcap application/vnd.tmobile-livetv tmo application/vnd.trid.tpt tpt application/vnd.triscape.mxs mxs @@ -458,7 +507,7 @@ application/vnd.umajin umj application/vnd.unity unityweb application/vnd.uoml+xml uoml application/vnd.vcx vcx -application/vnd.visio vsd vst vss vsw +application/vnd.visio vsd vss vst vsw application/vnd.visionary vis application/vnd.vsf vsf application/vnd.wap.wbxml wbxml @@ -482,6 +531,8 @@ application/vnd.yellowriver-custom-menu cmp application/vnd.zul zir zirz application/vnd.zzazz.deck+xml zaz application/voicexml+xml vxml +application/wasm wasm +application/watcherinfo+xml wif application/widget wgt application/winhlp hlp application/wsdl+xml wsdl @@ -490,15 +541,15 @@ application/x-7z-compressed 7z application/x-abiword abw application/x-ace-compressed ace application/x-arj arj -application/x-authorware-bin aab x32 u32 vox +application/x-authorware-bin aab u32 vox x32 application/x-authorware-map aam application/x-authorware-seg aas application/x-bcpio bcpio application/x-bittorrent torrent application/x-blorb blb blorb application/x-bzip bz -application/x-bzip2 bz2 boz -application/x-cbr cbr cba cbt cbz cb7 +application/x-bzip2 boz bz2 +application/x-cbr cb7 cba cbr cbt cbz application/x-cdlink vcd application/x-cfs-compressed cfs application/x-chat chat @@ -510,7 +561,7 @@ application/x-cpio cpio application/x-csh csh application/x-debian-package udeb application/x-dgc-compressed dgc -application/x-director dir dcr dxr cst cct cxt w3d fgd swa +application/x-director cct cst cxt dcr dir dxr fgd swa w3d application/x-doom wad application/x-dtbncx+xml ncx application/x-dtbook+xml dtb @@ -521,11 +572,9 @@ application/x-eva eva application/x-font-bdf bdf application/x-font-ghostscript gsf application/x-font-linux-psf psf -application/x-font-otf otf application/x-font-pcf pcf application/x-font-snf snf -application/x-font-ttf ttf ttc -application/x-font-type1 pfa pfb pfm afm +application/x-font-type1 afm pfa pfb pfm application/x-freearc arc application/x-futuresplash spl application/x-gca-compressed gca @@ -538,12 +587,13 @@ application/x-httpd-php php application/x-install-instructions install application/x-java-archive-diff jardiff application/x-java-jnlp-file jnlp +application/x-keepass2 kdbx application/x-latex latex application/x-lua-bytecode luac -application/x-lzh-compressed lzh lha +application/x-lzh-compressed lha lzh application/x-makeself run application/x-mie mie -application/x-mobipocket-ebook prc mobi +application/x-mobipocket-ebook mobi prc application/x-ms-application application application/x-ms-shortcut lnk application/x-ms-wmd wmd @@ -553,22 +603,21 @@ application/x-msaccess mdb application/x-msbinder obd application/x-mscardfile crd application/x-msclip clp -application/x-msdownload com bat -application/x-msmediaview mvb m13 m14 -application/x-msmetafile wmf emf emz +application/x-msdownload bat com +application/x-msmediaview m13 m14 mvb +application/x-msmetafile emf emz wmf application/x-msmoney mny application/x-mspublisher pub application/x-msschedule scd application/x-msterminal trm application/x-mswrite wri -application/x-netcdf nc cdf +application/x-netcdf cdf nc application/x-ns-proxy-autoconfig pac application/x-nzb nzb application/x-perl pl pm application/x-pkcs12 p12 pfx application/x-pkcs7-certificates p7b spc application/x-pkcs7-certreqresp p7r -application/x-rar-compressed rar application/x-redhat-package-manager rpm application/x-research-info-systems ris application/x-sea sea @@ -588,7 +637,7 @@ application/x-tar tar application/x-tcl tcl tk application/x-tex tex application/x-tex-tfm tfm -application/x-texinfo texinfo texi +application/x-texinfo texi texinfo application/x-tgif obj application/x-ustar ustar application/x-virtualbox-hdd hdd @@ -601,34 +650,40 @@ application/x-virtualbox-vhd vhd application/x-virtualbox-vmdk vmdk application/x-wais-source src application/x-web-app-manifest+json webapp -application/x-x509-ca-cert der crt pem +application/x-x509-ca-cert crt der pem application/x-xfig fig application/x-xliff+xml xlf application/x-xpinstall xpi application/x-xz xz application/x-zmachine z1 z2 z3 z4 z5 z6 z7 z8 application/xaml+xml xaml +application/xcap-att+xml xav +application/xcap-caps+xml xca application/xcap-diff+xml xdf +application/xcap-el+xml xel +application/xcap-ns+xml xns application/xenc+xml xenc -application/xhtml+xml xhtml xht -application/xml xml xsl xsd rng +application/xhtml+xml xht xhtml +application/xml rng xml xsd xsl application/xml-dtd dtd application/xop+xml xop application/xproc+xml xpl application/xslt+xml xslt application/xspf+xml xspf -application/xv+xml mxml xhvml xvml xvm +application/xv+xml mxml xhvml xvm xvml application/yang yang application/yin+xml yin application/zip zip audio/3gpp 3gpp audio/adpcm adp +audio/amr amr audio/basic au snd -audio/midi mid midi kar rmi +audio/midi kar mid midi rmi +audio/mobile-xmf mxmf audio/mp3 mp3 audio/mp4 m4a mp4a -audio/mpeg mpga mp2 mp2a m2a m3a -audio/ogg oga ogg spx +audio/mpeg m2a m3a mp2 mp2a mpga +audio/ogg oga ogg opus spx audio/s3m s3m audio/silk sil audio/vnd.dece.audio uva uvva @@ -645,14 +700,14 @@ audio/vnd.rip rip audio/wav wav audio/webm weba audio/x-aac aac -audio/x-aiff aif aiff aifc +audio/x-aiff aif aifc aiff audio/x-caf caf audio/x-flac flac audio/x-matroska mka audio/x-mpegurl m3u audio/x-ms-wax wax audio/x-ms-wma wma -audio/x-pn-realaudio ram ra +audio/x-pn-realaudio ra ram audio/x-pn-realaudio-plugin rmp audio/xm xm chemical/x-cdx cdx @@ -661,22 +716,57 @@ chemical/x-cmdf cmdf chemical/x-cml cml chemical/x-csml csml chemical/x-xyz xyz +font/collection ttc +font/otf otf +font/ttf ttf +font/woff woff +font/woff2 woff2 +image/aces exr image/apng apng +image/avci avci +image/avcs avcs +image/avif avif image/bmp bmp image/cgm cgm +image/dicom-rle drle +image/fits fits image/g3fax g3 image/gif gif +image/heic heic +image/heic-sequence heics +image/heif heif +image/heif-sequence heifs +image/hej2k hej2 +image/hsj2 hsj2 image/ief ief -image/jpeg jpeg jpg jpe +image/jls jls +image/jp2 jp2 jpg2 +image/jpeg jpe jpeg jpg +image/jph jph +image/jphc jhc +image/jpm jpm +image/jpx jpf jpx +image/jxr jxr +image/jxra jxra +image/jxrs jxrs +image/jxs jxs +image/jxsc jxsc +image/jxsi jxsi +image/jxss jxss image/ktx ktx +image/ktx2 ktx2 image/png png image/prs.btif btif +image/prs.pti pti image/sgi sgi image/svg+xml svg svgz -image/tiff tiff tif +image/t38 t38 +image/tiff tif tiff +image/tiff-fx tfx image/vnd.adobe.photoshop psd -image/vnd.dece.graphic uvi uvvi uvg uvvg -image/vnd.djvu djvu djv +image/vnd.airzip.accelerator.azv azv +image/vnd.dece.graphic uvg uvi uvvg uvvi +image/vnd.djvu djv djvu image/vnd.dvb.subtitle sub image/vnd.dwg dwg image/vnd.dxf dxf @@ -685,21 +775,25 @@ image/vnd.fpx fpx image/vnd.fst fst image/vnd.fujixerox.edmics-mmr mmr image/vnd.fujixerox.edmics-rlc rlc +image/vnd.microsoft.icon ico +image/vnd.ms-dds dds image/vnd.ms-modi mdi image/vnd.ms-photo wdp image/vnd.net-fpx npx +image/vnd.pco.b16 b16 +image/vnd.tencent.tap tap +image/vnd.valve.source.texture vtf image/vnd.wap.wbmp wbmp image/vnd.xiff xif +image/vnd.zbrush.pcx pcx image/webp webp image/x-3ds 3ds image/x-cmu-raster ras image/x-cmx cmx -image/x-freehand fh fhc fh4 fh5 fh7 -image/x-icon ico +image/x-freehand fh fh4 fh5 fh7 fhc image/x-jng jng image/x-mrsid-image sid -image/x-pcx pcx -image/x-pict pic pct +image/x-pict pct pic image/x-portable-anymap pnm image/x-portable-bitmap pbm image/x-portable-graymap pgm @@ -709,18 +803,35 @@ image/x-tga tga image/x-xbitmap xbm image/x-xpixmap xpm image/x-xwindowdump xwd +message/disposition-notification disposition-notification +message/global u8msg +message/global-delivery-status u8dsn +message/global-disposition-notification u8mdn +message/global-headers u8hdr message/rfc822 eml mime +message/vnd.wfa.wsc wsc +model/3mf 3mf model/gltf+json gltf model/gltf-binary glb -model/iges igs iges -model/mesh msh mesh silo +model/iges iges igs +model/mesh mesh msh silo +model/mtl mtl +model/step+xml stpx +model/step+zip stpz +model/step-xml+zip stpxz model/vnd.collada+xml dae model/vnd.dwf dwf model/vnd.gdl gdl model/vnd.gtw gtw model/vnd.mts mts +model/vnd.opengex ogex +model/vnd.parasolid.transmit.binary x_b +model/vnd.parasolid.transmit.text x_t +model/vnd.sap.vds vds +model/vnd.usdz+zip usdz +model/vnd.valve.source.compiled-map bsp model/vnd.vtu vtu -model/vrml wrl vrml +model/vrml vrml wrl model/x3d+binary x3db x3dbz model/x3d+vrml x3dv x3dvz model/x3d+xml x3d x3dz @@ -729,22 +840,24 @@ text/calendar ics if text/coffeescript coffee litcoffee text/css css text/csv csv -text/hjson hjson -text/html html htm shtml +text/html htm html shtml text/jade jade text/jsx jsx text/less less text/markdown markdown md text/mathml mml +text/mdx mdx text/n3 n3 -text/plain txt text conf def list log in ini +text/plain conf def in ini list log text txt text/prs.lines.tag dsc text/richtext rtx -text/sgml sgml sgm +text/sgml sgm sgml +text/shex shex text/slim slim slm -text/stylus stylus styl +text/spdx spdx +text/stylus styl stylus text/tab-separated-values tsv -text/troff t tr roff man me ms +text/troff man me ms roff t tr text/turtle ttl text/uri-list uri uris urls text/vcard vcard @@ -752,6 +865,7 @@ text/vnd.curl curl text/vnd.curl.dcurl dcurl text/vnd.curl.mcurl mcurl text/vnd.curl.scurl scurl +text/vnd.familysearch.gedcom ged text/vnd.fly fly text/vnd.fmi.flexstor flx text/vnd.graphviz gv @@ -761,10 +875,10 @@ text/vnd.sun.j2me.app-descriptor jad text/vnd.wap.wml wml text/vnd.wap.wmlscript wmls text/vtt vtt -text/x-asm s asm -text/x-c c cc cxx cpp h hh dic +text/x-asm asm s +text/x-c c cc cpp cxx dic h hh text/x-component htc -text/x-fortran f for f77 f90 +text/x-fortran f f77 f90 for text/x-handlebars-template hbs text/x-java-source java text/x-lua lua @@ -787,14 +901,15 @@ video/3gpp2 3g2 video/h261 h261 video/h263 h263 video/h264 h264 +video/iso.segment m4s video/jpeg jpgv -video/jpm jpm jpgm +video/jpm jpgm video/mj2 mj2 mjp2 video/mp2t ts video/mp4 mp4 mp4v mpg4 -video/mpeg mpeg mpg mpe m1v m2v +video/mpeg m1v m2v mpe mpeg mpg video/ogg ogv -video/quicktime qt mov +video/quicktime mov qt video/vnd.dece.hd uvh uvvh video/vnd.dece.mobile uvm uvvm video/vnd.dece.pd uvp uvvp @@ -802,7 +917,7 @@ video/vnd.dece.sd uvs uv video/vnd.dece.video uvv uvvv video/vnd.dvb.file dvb video/vnd.fvt fvt -video/vnd.mpegurl mxu m4u +video/vnd.mpegurl m4u mxu video/vnd.ms-playready.media.pyv pyv video/vnd.uvvu.mp4 uvu uvvu video/vnd.vivo viv @@ -811,7 +926,7 @@ video/x-f4v f4v video/x-fli fli video/x-flv flv video/x-m4v m4v -video/x-matroska mkv mk3d mks +video/x-matroska mk3d mks mkv video/x-mng mng video/x-ms-asf asf asx video/x-ms-vob vob diff --git a/lib/jekyll/page.rb b/lib/jekyll/page.rb index caea9cd924e..b8b8e4492ff 100644 --- a/lib/jekyll/page.rb +++ b/lib/jekyll/page.rb @@ -183,5 +183,20 @@ def trigger_hooks(hook_name, *args) def write? true end + + # The Page excerpt_separator, from the YAML Front-Matter or site + # default excerpt_separator value + # + # Returns the document excerpt_separator + def excerpt_separator + (data["excerpt_separator"] || site.config["excerpt_separator"]).to_s + end + + # Whether to generate an excerpt + # + # Returns true if the excerpt separator is configured. + def generate_excerpt? + !excerpt_separator.empty? + end end end diff --git a/lib/jekyll/readers/data_reader.rb b/lib/jekyll/readers/data_reader.rb index 445d548802b..ebd2f67adbd 100644 --- a/lib/jekyll/readers/data_reader.rb +++ b/lib/jekyll/readers/data_reader.rb @@ -54,16 +54,14 @@ def read_data_to(dir, data) def read_data_file(path) case File.extname(path).downcase when ".csv" - CSV.read(path, { + CSV.read(path, :headers => true, - :encoding => site.config["encoding"], - }).map(&:to_hash) + :encoding => site.config["encoding"]).map(&:to_hash) when ".tsv" - CSV.read(path, { + CSV.read(path, :col_sep => "\t", :headers => true, - :encoding => site.config["encoding"], - }).map(&:to_hash) + :encoding => site.config["encoding"]).map(&:to_hash) else SafeYAML.load_file(path) end diff --git a/lib/jekyll/site.rb b/lib/jekyll/site.rb index 25257a64c0d..44bacc9047c 100644 --- a/lib/jekyll/site.rb +++ b/lib/jekyll/site.rb @@ -314,15 +314,15 @@ def relative_permalinks_are_deprecated # # Returns an Array of Documents which should be written def docs_to_write - @docs_to_write ||= documents.select(&:write?) + documents.select(&:write?) end # Get all the documents # # Returns an Array of all Documents def documents - collections.reduce(Set.new) do |docs, (_, collection)| - docs + collection.docs + collection.files + collections.each_with_object(Set.new) do |(_, collection), set| + set.merge(collection.docs).merge(collection.files) end.to_a end diff --git a/lib/jekyll/stevenson.rb b/lib/jekyll/stevenson.rb index bbec66171f6..5ab4495ce95 100644 --- a/lib/jekyll/stevenson.rb +++ b/lib/jekyll/stevenson.rb @@ -3,13 +3,10 @@ module Jekyll class Stevenson < ::Logger def initialize - @progname = nil - @level = DEBUG - @default_formatter = Formatter.new - @logdev = $stdout - @formatter = proc do |_, _, _, msg| + formatter = proc do |_, _, _, msg| msg.to_s end + super($stdout, :formatter => formatter) end def add(severity, message = nil, progname = nil) diff --git a/lib/jekyll/tags/include.rb b/lib/jekyll/tags/include.rb index 0ee983dc177..49339b1d86c 100644 --- a/lib/jekyll/tags/include.rb +++ b/lib/jekyll/tags/include.rb @@ -22,7 +22,7 @@ class IncludeTag < Liquid::Tag !mx FULL_VALID_SYNTAX = %r!\A\s*(?:#{VALID_SYNTAX}(?=\s|\z)\s*)*\z! - VALID_FILENAME_CHARS = %r!^[\w/\.-]+$! + VALID_FILENAME_CHARS = %r!^[\w/.\-()+~\#@]+$! INVALID_SEQUENCES = %r![./]{2,}! def initialize(tag_name, markup, tokens) @@ -147,12 +147,17 @@ def render(context) end def add_include_to_dependency(site, path, context) - if context.registers[:page] && context.registers[:page].key?("path") - site.regenerator.add_dependency( - site.in_source_dir(context.registers[:page]["path"]), - path - ) - end + page = context.registers[:page] + return unless page + return unless page.key?("path") + + absolute_path = \ + if page["collection"] + site.in_source_dir(site.config["collections_dir"], page["path"]) + else + site.in_source_dir(page["path"]) + end + site.regenerator.add_dependency(absolute_path, path) end def load_cached_partial(path, context) @@ -191,7 +196,7 @@ def realpath_prefixed_with?(path, dir) # This method allows to modify the file content by inheriting from the class. def read_file(file, context) - File.read(file, file_read_opts(context)) + File.read(file, **file_read_opts(context)) end private @@ -224,6 +229,7 @@ def page_path(context) else File.join(site.config["collections_dir"], page_payload["path"]) end + resource_path.sub!(%r!/#excerpt\z!, "") site.in_source_dir File.dirname(resource_path) end end diff --git a/lib/jekyll/theme.rb b/lib/jekyll/theme.rb index a21b4bdfea1..8b28fbb47dc 100644 --- a/lib/jekyll/theme.rb +++ b/lib/jekyll/theme.rb @@ -56,12 +56,28 @@ def path_for(folder) end def realpath_for(folder) - File.realpath(Jekyll.sanitized_path(root, folder.to_s)) - rescue Errno::ENOENT, Errno::EACCES, Errno::ELOOP - Jekyll.logger.warn "Invalid theme folder:", folder + # This resolves all symlinks for the theme subfolder and then ensures + # that the directory remains inside the theme root. This prevents the + # use of symlinks for theme subfolders to escape the theme root. + # However, symlinks are allowed to point to other directories within the theme. + Jekyll.sanitized_path(root, File.realpath(Jekyll.sanitized_path(root, folder.to_s))) + rescue Errno::ENOENT, Errno::EACCES, Errno::ELOOP => e + log_realpath_exception(e, folder) nil end + def log_realpath_exception(err, folder) + return if err.is_a?(Errno::ENOENT) + + case err + when Errno::EACCES + Jekyll.logger.error "Theme error:", "Directory '#{folder}' is not accessible." + when Errno::ELOOP + Jekyll.logger.error "Theme error:", + "Directory '#{folder}' includes a symbolic link loop." + end + end + def gemspec @gemspec ||= Gem::Specification.find_by_name(name) rescue Gem::LoadError diff --git a/lib/jekyll/utils/platforms.rb b/lib/jekyll/utils/platforms.rb index 6665527a726..0748b0e3c4a 100644 --- a/lib/jekyll/utils/platforms.rb +++ b/lib/jekyll/utils/platforms.rb @@ -70,9 +70,9 @@ def linux? private def proc_version @proc_version ||= begin - Pathutil.new( + File.read( "/proc/version" - ).read + ).downcase rescue Errno::ENOENT nil end diff --git a/lib/jekyll/utils/win_tz.rb b/lib/jekyll/utils/win_tz.rb index 239650f71fe..ad277d3a6a5 100644 --- a/lib/jekyll/utils/win_tz.rb +++ b/lib/jekyll/utils/win_tz.rb @@ -11,64 +11,35 @@ module WinTZ # timezone - the IANA Time Zone specified in "_config.yml" # # Returns a string that ultimately re-defines ENV["TZ"] in Windows - def calculate(timezone) + def calculate(timezone, now = Time.now) External.require_with_graceful_fail("tzinfo") unless defined?(TZInfo) tz = TZInfo::Timezone.get(timezone) - difference = Time.now.to_i - tz.now.to_i + + # + # Use period_for_utc and utc_total_offset instead of + # period_for and observed_utc_offset for compatibility with tzinfo v1. + offset = tz.period_for_utc(now.getutc).utc_total_offset + # # POSIX style definition reverses the offset sign. # e.g. Eastern Standard Time (EST) that is 5Hrs. to the 'west' of Prime Meridian # is denoted as: # EST+5 (or) EST+05:00 - # Reference: http://www.gnu.org/software/libc/manual/html_node/TZ-Variable.html - sign = difference < 0 ? "-" : "+" - offset = sign == "-" ? "+" : "-" unless difference.zero? - # - # convert the difference (in seconds) to hours, as a rational number, and perform - # a modulo operation on it. - modulo = modulo_of(rational_hour(difference)) + # Reference: https://www.gnu.org/software/libc/manual/html_node/TZ-Variable.html + sign = offset.positive? ? "-" : "+" + + rational_hours = offset.abs.to_r / 3600 + hours = rational_hours.to_i + minutes = ((rational_hours - hours) * 60).to_i + # - # Format the hour as a two-digit number. - # Establish the minutes based on modulo expression. - hh = format("%02d", absolute_hour(difference).ceil) - mm = modulo.zero? ? "00" : "30" + # Format the hours and minutes as two-digit numbers. + time = format("%(«|«)Pit(›|›)hy(»|»)<\/p>!, \ - markdown.convert(%("Pit'hy")).strip + converter.convert(%("Pit'hy")).strip end end - should "render fenced code blocks with syntax highlighting" do - result = nokogiri_fragment(@markdown.convert(Utils.strip_heredoc(<<-MARKDOWN))) - ~~~ruby - puts "Hello World" - ~~~ - MARKDOWN - div_highlight = "" - div_highlight = ">div.highlight" unless Utils::Rouge.old_api? - selector = "div.highlighter-rouge#{div_highlight}>pre.highlight>code" - refute result.css(selector).empty? - end - context "when a custom highlighter is chosen" do should "use the chosen highlighter if it's available" do override = { "highlighter" => nil, - "markdown" => "kramdown", "kramdown" => { - "syntax_highlighter" => :coderay, + "syntax_highlighter" => "coderay", }, } - markdown = Converters::Markdown.new(Utils.deep_merge_hashes(@config, override)) - result = nokogiri_fragment(markdown.convert(Utils.strip_heredoc(<<-MARKDOWN))) + converter = fixture_converter(Utils.deep_merge_hashes(@config, override)) + result = nokogiri_fragment(converter.convert(<<~MARKDOWN)) ~~~ruby puts "Hello World" ~~~ @@ -120,7 +162,6 @@ class TestKramdown < JekyllUnitTest should "support legacy enable_coderay... for now" do override = { - "markdown" => "kramdown", "kramdown" => { "enable_coderay" => true, }, @@ -128,8 +169,9 @@ class TestKramdown < JekyllUnitTest @config.delete("highlighter") @config["kramdown"].delete("syntax_highlighter") - markdown = Converters::Markdown.new(Utils.deep_merge_hashes(@config, override)) - result = nokogiri_fragment(markdown.convert(Utils.strip_heredoc(<<-MARKDOWN))) + + converter = fixture_converter(Utils.deep_merge_hashes(@config, override)) + result = nokogiri_fragment(converter.convert(<<~MARKDOWN)) ~~~ruby puts "Hello World" ~~~ @@ -141,24 +183,26 @@ class TestKramdown < JekyllUnitTest end should "move coderay to syntax_highlighter_opts" do - original = Kramdown::Document.method(:new) - markdown = Converters::Markdown.new(Utils.deep_merge_hashes(@config, { + override = { "higlighter" => nil, - "markdown" => "kramdown", "kramdown" => { "syntax_highlighter" => "coderay", "coderay" => { "hello" => "world", }, }, - })) + } + original = Kramdown::Document.method(:new) + converter = fixture_converter( + Utils.deep_merge_hashes(@config, override) + ) expect(Kramdown::Document).to receive(:new) do |arg1, hash| assert_equal hash["syntax_highlighter_opts"]["hello"], "world" original.call(arg1, hash) end - markdown.convert("hello world") + converter.convert("hello world") end end end diff --git a/test/test_layout_reader.rb b/test/test_layout_reader.rb index 96d2045a148..805bbac8747 100644 --- a/test/test_layout_reader.rb +++ b/test/test_layout_reader.rb @@ -31,5 +31,52 @@ class TestLayoutReader < JekyllUnitTest assert_equal LayoutReader.new(@site).layout_directory, source_dir("blah/_layouts") end end + + context "when a layout is a symlink" do + setup do + symlink_if_allowed("/etc/passwd", source_dir("_layouts", "symlink.html")) + + @site = fixture_site( + "safe" => true, + "include" => ["symlink.html"] + ) + end + + teardown do + FileUtils.rm_f(source_dir("_layouts", "symlink.html")) + end + + should "only read the layouts which are in the site" do + skip_if_windows "Jekyll does not currently support symlinks on Windows." + + layouts = LayoutReader.new(@site).read + + refute layouts.key?("symlink"), "Should not read the symlinked layout" + end + end + + context "with a theme" do + setup do + symlink_if_allowed("/etc/passwd", theme_dir("_layouts", "theme-symlink.html")) + @site = fixture_site( + "include" => ["theme-symlink.html"], + "theme" => "test-theme", + "safe" => true + ) + end + + teardown do + FileUtils.rm_f(theme_dir("_layouts", "theme-symlink.html")) + end + + should "not read a symlink'd theme" do + skip_if_windows "Jekyll does not currently support symlinks on Windows." + + layouts = LayoutReader.new(@site).read + + refute layouts.key?("theme-symlink"), \ + "Should not read symlinked layout from theme" + end + end end end diff --git a/test/test_rdiscount.rb b/test/test_rdiscount.rb index 7116c29a238..796ee81dda6 100644 --- a/test/test_rdiscount.rb +++ b/test/test_rdiscount.rb @@ -28,16 +28,16 @@ class TestRdiscount < JekyllUnitTest should "render toc" do toc = <<-TOC - +
) +
%(1\n | ) +
- %(test | test\n | ) +
%(
This is not yet highlighted
\n1
-
test
test\nThis should not be highlighted, right?
EOS assert_match(expected, @result) @@ -1093,6 +1093,49 @@ def highlight_block_with_opts(options_string) end end + context "with include file with special characters without params" do + setup do + content = <<~CONTENT + --- + title: special characters + --- + + {% include params@2.0.html %} + CONTENT + create_post(content, + "permalink" => "pretty", + "source" => source_dir, + "destination" => dest_dir, + "read_posts" => true) + end + + should "include file with empty parameters" do + assert_match "", @result + end + end + + context "with include file with special characters with params" do + setup do + content = <<~CONTENT + --- + title: special characters + --- + + {% include params@2.0.html param1="foobar" param2="bazbar" %} + CONTENT + create_post(content, + "permalink" => "pretty", + "source" => source_dir, + "destination" => dest_dir, + "read_posts" => true) + end + + should "include file with empty parameters" do + assert_match "