Go Scanner #28
Go Scanner #28
Conversation
Draft version, copy from c
|
Thanks! We need some example code to test it. Can you point me to a good source for Go example code, especially using stranger syntax? |
|
Maybe we could slurp up part of the standard library? Or are you looking for a one file example that essentially uses every construct available? Fyi, the spec is here: http://golang.org/ref/spec |
That would be awesome. |
|
How about this one, just a part of Go tutorial exercise, a web crawler demo On Sat, Jul 28, 2012 at 4:10 AM, Kornelius Kalnbach <
Eric Guo Tel :+86-21-60905910 |
|
@Eric-Guo which one? |
|
This one |
|
Looks like a good start. It has channels and anonymous functions. It would be good to have the syntax for Type assertions and Composite literals, since those look a bit funky. Another one I was thinking is when you pass a channel to a function and it only allows incoming or only outgoing channels, just to be sure the little <- arrow appears appropriately in that context. Fyi, this is the Pygments Lexer for Go, though I don't think it highlights a whole lot. Once we have support for Go, do you think it would also be worthwhile to add Go templates? |
|
Cgo is also a weird thing, you basically embed C code into a file, though in a comment block? Not sure if there is a nice way to reuse that highlighter. |
It's possible, the PHP and ERb do this. I guess we need to keep track of the regexp.MustCompile(`http://baike.baidu.com/view/[/\w\.-]+`)The spec says:
So we need to handle this one differently, like Ruby's single quotes. |
|
nathany:
Is there anything specific you miss in the Pygments version? Usually, their lexers are quite good. |
|
I added the fetcher example and @AlanQuatermain's go-apns to test suite: |
|
GitHub uses Pygments, and when I look at Go code here, such as go-apns, it just looks plain compared to Sublime Text. Of course I tend to use dark themes, so I tried to find a theme that is closer to GitHub to pull out the differences: https://www.dropbox.com/s/0dzhtm6th7olzhp/sublime-vs-pygments.png There is definitely more variation here. The receiver (e *Error) is highlighted, the function names are a different color, the short assignment := operator is different than other operators, etc. Ideally we'll get more themes built into CodeRay, such as Twilight, but that's a separate topic. |
|
If it's of any use, you might take a look at the TextMate bundle I wrote for Go, which parses out a lot of these types for syntax highlighting: https://github.com/AlanQuatermain/go-tmbundle/blob/master/Syntaxes/Go.tmLanguage Also: I finished go-apns? Damn, I thought that was just a placeholder with an empty function… Yeah, it's been a while… |
|
nathany:
Thanks for the screenshots, that's helpful! But I'm not convinced that the Sublime coloring is better. Why is I wouldn't rate the quality of a highlighter by the number of different colors you get ;) For example, most editors highlight common Ruby methods like Sorry for the rant. My belief is that a highlighter should only highlight differntly what it can definitely distinguish. Since we're not parsing (just lexing), there's a hard limit for this. |
|
The := operator is a special case: it initializes the left-hand side to the type and value of the right-hand side. It's only usable for initialization of a variable, not assignment. As such, I'd say it's important to call it out with syntax highlighting. Sent from my iPhone On 2012-07-28, at 3:47 PM, Kornelius Kalnbach reply@reply.github.com wrote:
|
|
Ah, I see…so |
|
Yep, it means 'initialize with', and when using it you don't need to declare the type of the variable. These two are functionally identical: var myString string = "something" Sent from my iPhone On 2012-07-28, at 4:02 PM, Kornelius Kalnbach reply@reply.github.com wrote:
|
|
I don't know if all the highlighting Sublime is doing is meaningful. ^_^ I'm all for having smarter highlighting...
I don't think that's overkill, while still being much more colourful than GitHub/Pygments, which appears to only be highlighting keywords, numeric/string literals, and built in types: package main
import (
"fmt"
"labix.org/v2/mgo"
"labix.org/v2/mgo/bson"
)
type Person struct {
Name string
Phone string
}
func main() {
session, err := mgo.Dial("localhost")
if err != nil {
panic(err)
}
defer session.Close()
session.SetMode(mgo.Monotonic, true)
c := session.DB("test").C("people")
err = c.Insert(
&Person{"Ale", "+55 53 8116 9639"},
&Person{"Cla", "+55 53 8402 8510"})
if err != nil {
panic(err)
}
result := Person{}
err = c.Find(bson.M{"name": "Ale"}).One(&result)
if err != nil {
panic(err)
}
fmt.Println("Phone:", result.Phone)
} |
|
Here's one more screenshot to help clarify what it's doing: At the bottom you see Not entirely sure if that's good or bad. Depends on the value of that distinction, I suppose. Maybe it's not that helpful. It certainly seem like using the sa
8000
me colors to mean different things isn't a great idea (yellow for Anyway, my whole point is that we can probably do better than what Pygments currently offers for Go, not that we should copy Sublime Text. In fact, I might need to build a custom color theme for Sublime to match what we come up with. Nathan. |
|
bump Is this coming along? Need any help? |
I guess so. But I have not enough knowledge of Go to do that. So, do we include the scanner as-is in CodeRay 1.1, or do we keep working on it? If so, who implements the improvements? |
|
@korny I think this pull request was a very rough draft based on the C code? I'm not familiar with writing scanners, so I get pretty lost in the If you want to take a stab at it, the Go language specification is pretty readable, and the grammar is fairly small. The scanner that Go has built in may also be a good reference. You already have Go sample code, but I will work on some that tries to exercise the whole grammar, with comments to help. |
|
I made some updates/fixes. It's presentable now (I think), but not great (as you said). It would be awesome (and quite CodeRay style) to make a new scanner from scratch after the official scanner. But I think it's a lot of work, turing the char-by-char lexer into a readable regexp-based scanner. I don't have the time to do it myself, and I'd rather work on the Objective-C scanner (planned for 1.2). So, let's vote. Merge the current version from |
That would be great in any case. |
|
@korny Do you want all the sample code to be free of errors, at least that it compiles? I just fixed the apns.go sample. Oh, and I see you added some *.raydebug files to that repo. Do you just copy them across after running |
I added a nicer API,
No ^^ I don't even have a compiler here. |
|
Rouge has an interesting way of testing the highlighter, though this isn't exactly valid code. What do you think? |
Interesting idea. I don't think it's a replacement to real example code, but a very nice addition to the usual examples. |
Draft version, just copy from c and add keywords based on http://golang.org/ref/spec
(Edit @korny: Fixed the title)