Skip to content
This repository was archived by the owner on Apr 1, 2025. It is now read-only.

Commit 0d78391

Browse files
author
Patrick Thomson
authored
Merge branch 'master' into lingo
2 parents 691f1fd + b7a52b4 commit 0d78391

File tree

14 files changed

+124
-8
lines changed

14 files changed

+124
-8
lines changed

‎docs/adding-new-languages.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@ Please note that this list of steps reflects the state of Semantic as is, not wh
77
## The procedure
88

99
1. **Find or write a [tree-sitter](https://tree-sitter.github.io) parser for your language.** The tree-sitter [organization page](https://github.com/tree-sitter) has a number of parsers beyond those we currently support in Semantic; look there first to make sure you're not duplicating work. The tree-sitter [documentation on creating parsers](http://tree-sitter.github.io/tree-sitter/creating-parsers) provides an exhaustive look at the process of developing and debugging tree-sitter parsers. Though we do not support grammars written with other toolkits such as [ANTLR](https://www.antlr.org), translating an ANTLR or other BNF-style grammar into a tree-sitter grammar is usually straightforward.
10-
2. **Create a Haskell library providing an interface to that C source.** The [`haskell-tree-sitter`](https://github.com/tree-sitter/haskell-tree-sitter/tree/master/languages) repository provides a Cabal package for each supported language. You can find an example of a pull request to add such a package here. Each package needs to provide two API surfaces:
10+
2. **Create a Haskell library providing an interface to that C source.** The [`haskell-tree-sitter`](https://github.com/tree-sitter/haskell-tree-sitter) repository provides a Cabal package for each supported language. You can find an example of a pull request to add such a package here. Each package needs to provide two API surfaces:
1111
* a bridged (via the FFI) reference to the toplevel parser in the generated file ([example](https://github.com/tree-sitter/haskell-tree-sitter/blob/master/tree-sitter-json/internal/TreeSitter/JSON/Internal.hs))
1212
* symbol datatypes for each syntax node in the parser, generated with the `mkSymbolDatatype` Template Haskell splice ([example](https://github.com/tree-sitter/haskell-tree-sitter/blob/master/tree-sitter-json/TreeSitter/JSON.hs))
1313
3. **Identify the new syntax nodes required to represent your language.** While we provide an extensive library of reusable AST nodes for [literals](https://github.com/github/semantic/blob/master/src/Data/Syntax/Literal.hs), [expressions](https://github.com/github/semantic/blob/master/src/Data/Syntax/Expression.hs), [statements](https://github.com/github/semantic/blob/master/src/Data/Syntax/Statement.hs), and [types](https://github.com/github/semantic/blob/master/src/Data/Syntax/Type.hs), most languages will require some syntax nodes not found in other languages. You'll need to create a new module providing those data types, and those data types must be written as an open union: [here](https://github.com/github/semantic/commits/master/src/Language/Ruby/Syntax.hs?author=charliesome) is an example for Ruby's syntactic details.

‎src/Language/TSX/Assignment.hs

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -429,7 +429,7 @@ jsxAttribute' :: Assignment Term
429429
jsxAttribute' = jsxAttribute <|> jsxExpression'
430430

431431
jsxOpeningElement' :: Assignment Term
432-
jsxOpeningElement' = makeTerm <$> symbol Grammar.JsxOpeningElement <*> children (TSX.Syntax.JsxOpeningElement <$> term jsxElementName <*> manyTerm jsxAttribute')
432+
jsxOpeningElement' = makeTerm <$> symbol Grammar.JsxOpeningElement <*> children (TSX.Syntax.JsxOpeningElement <$> term jsxElementName <*> term (typeArguments' <|> emptyTerm) <*> manyTerm jsxAttribute')
433433

434434
jsxElementName :: Assignment Term
435435
jsxElementName = choice [ identifier, nestedIdentifier, jsxNamespaceName ]

‎src/Language/TSX/Syntax/JSX.hs

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -28,7 +28,7 @@ newtype JsxExpression a = JsxExpression { jsxExpression :: a }
2828

2929
instance Evaluatable JsxExpression
3030

31-
data JsxOpeningElement a = JsxOpeningElement { jsxOpeningElementIdentifier :: !a, jsxAttributes :: ![a] }
31+
data JsxOpeningElement a = JsxOpeningElement { jsxOpeningElementIdentifier :: !a, jsxOpeningElementTypeArguments :: a, jsxAttributes :: ![a] }
3232
deriving (Declarations1, Diffable, Eq, Foldable, FreeVariables1, Functor, Generic1, Hashable1, NFData1, Ord, Show, ToJSONFields1, Traversable)
3333
deriving (Eq1, Show1, Ord1) via Generically JsxOpeningElement
3434

‎src/Parsing/Parser.hs

Lines changed: 6 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -191,9 +191,13 @@ someASTParser :: Language -> Maybe SomeASTParser
191191
someASTParser Go = Just (SomeASTParser (ASTParser tree_sitter_go :: Parser (AST [] Go.Grammar)))
192192
someASTParser Haskell = Just (SomeASTParser (ASTParser tree_sitter_haskell :: Parser (AST [] Haskell.Grammar)))
193193
someASTParser Java = Just (SomeASTParser (ASTParser tree_sitter_java :: Parser (AST [] Java.Grammar)))
194-
someASTParser JavaScript = Just (SomeASTParser (ASTParser tree_sitter_typescript :: Parser (AST [] TypeScript.Grammar)))
195194
someASTParser JSON = Just (SomeASTParser (ASTParser tree_sitter_json :: Parser (AST [] JSON.Grammar)))
196-
someASTParser JSX = Just (SomeASTParser (ASTParser tree_sitter_typescript :: Parser (AST [] TypeScript.Grammar)))
195+
196+
-- Use the TSX parser for `.js` and `.jsx` files in case they use Flow type-annotation syntax.
197+
-- The TSX and Flow syntaxes are the same, whereas the normal TypeScript syntax is different.
198+
someASTParser JavaScript = Just (SomeASTParser (ASTParser tree_sitter_tsx :: Parser (AST [] TSX.Grammar)))
199+
someASTParser JSX = Just (SomeASTParser (ASTParser tree_sitter_tsx :: Parser (AST [] TSX.Grammar)))
200+
197201
someASTParser Python = Just (SomeASTParser (ASTParser tree_sitter_python :: Parser (AST [] Python.Grammar)))
198202
someASTParser Ruby = Just (SomeASTParser (ASTParser tree_sitter_ruby :: Parser (AST [] Ruby.Grammar)))
199203
someASTParser TypeScript = Just (SomeASTParser (ASTParser tree_sitter_typescript :: Parser (AST [] TypeScript.Grammar)))

‎test/fixtures/javascript/corpus/jsx.A.js

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
function Something() {
2+
return <div>
3+
<Foo>hello</Foo>
4+
</div>;
5+
}

‎test/fixtures/javascript/corpus/jsx.B.js

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
function Something() {
2+
return <div>
3+
<Foo<T>>goodbye</Foo>
4+
</div>;
5+
}

‎test/fixtures/javascript/corpus/jsx.diffA-B.txt

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
1+
(Statements
2+
(Function
3+
(Empty)
4+
(Empty)
5+
(Identifier)
6+
(StatementBlock
7+
(Return
8+
(JsxElement
9+
(JsxOpeningElement
10+
(Identifier)
11+
(Empty))
12+
(JsxText)
13+
(JsxElement
14+
(JsxOpeningElement
15+
(Identifier)
16+
{ (Empty)
17+
->(TypeArguments
18+
{+(TypeIdentifier)+}) })
19+
{ (JsxText)
20+
->(JsxText) }
21+
(JsxClosingElement
22+
(Identifier)))
23+
(JsxText)
24+
(JsxClosingElement
25+
(Identifier)))))))

‎test/fixtures/javascript/corpus/jsx.diffB-A.txt

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
1+
(Statements
2+
(Function
3+
(Empty)
4+
(Empty)
5+
(Identifier)
6+
(StatementBlock
7+
(Return
8+
(JsxElement
9+
(JsxOpeningElement
10+
(Identifier)
11+
(Empty))
12+
(JsxText)
13+
(JsxElement
14+
(JsxOpeningElement
15+
(Identifier)
16+
{ (TypeArguments
17+
{-(TypeIdentifier)-})
18+
->(Empty) })
19+
{ (JsxText)
20+
->(JsxText) }
21+
(JsxClosingElement
22+
(Identifier)))
23+
(JsxText)
24+
(JsxClosingElement
25+
(Identifier)))))))

‎test/fixtures/javascript/corpus/jsx.parseA.txt

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,22 @@
1+
(Statements
2+
(Function
3+
(Empty)
4+
(Empty)
5+
(Identifier)
6+
(StatementBlock
7+
(Return
8+
(JsxElement
9+
(JsxOpeningElement
10+
(Identifier)
11+
(Empty))
12+
(JsxText)
13+
(JsxElement
14+
(JsxOpeningElement
15+
(Identifier)
16+
(Empty))
17+
(JsxText)
18+
(JsxClosingElement
19+
(Identifier)))
20+
(JsxText)
21+
(JsxClosingElement
22+
(Identifier)))))))

‎test/fixtures/javascript/corpus/jsx.parseB.txt

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,23 @@
1+
(Statements
2+
(Function
3+
(Empty)
4+
(Empty)
5+
(Identifier)
6+
(StatementBlock
7+
(Return
8+
(JsxElement
9+
(JsxOpeningElement
10+
(Identifier)
11+
(Empty))
12+
(JsxText)
13+
(JsxElement
14+
(JsxOpeningElement
15+
(Identifier)
16+
(TypeArguments
17+
(TypeIdentifier)))
18+
(JsxText)
19+
(JsxClosingElement
20+
(Identifier)))
21+
(JsxText)
22+
(JsxClosingElement
23+
(Identifier)))))))

0 commit comments

Comments
 (0)