37 Commits

Author SHA1 Message Date
Łukasz Rogalski
b5ef93c06e Prework for introducing mypyc (#6433) 2024-11-09 19:10:10 +00:00
Adam Weeden
92db7ffcbd Add volume syntax support for Databricks (#6179) 2024-09-13 11:58:56 +00:00
j-svensmark
1e2e54e1fd Parent dir config search (#5958) 2024-06-26 10:56:15 +00:00
Germain Lefebvre
720ab7f7f1 fix(dialect-clickhouse): Support DROP statements (#4821) 2023-04-27 07:39:02 +00:00
Danny Jones
f462e996a5 Parser Performance: Cache segment string repr to reduce function calls (#4278)
* Cache segment string repr to reduce function calls

* Cache segment string repr to reduce function calls

* Add helpful comment

* linting

Co-authored-by: Alan Cruickshank <alanmcruickshank@gmail.com>
2023-01-15 22:37:40 +00:00
Alan Cruickshank
0b21edbc80 Enable dumping of performance information to csv. (#3937)
* Persist timing information

* beta messaging

* fail nicely for files which don't parse

Co-authored-by: Barry Hart <barrywhart@yahoo.com>
2022-10-09 14:35:33 -04:00
Michael O Toole
8126c06720 L003 Refactor V2 (#2884)
* Calculate Tempalte info with Line using class

* _LineSummary for the win

* _Memory joins the team

* L003 preformance improvements

* Correct Comment Errors

* Coverage fixes

* linting errors

* memo is_empty

* Fix coverage

* linting Issue

* Seperate out more sub elements

* Hanging into method

* Running Again

* further clean up

* spelling Errors

* Comments spelling and fStrings

* Linting Bugs

* Fixed now incorrect tests

* remove unneeded test

* remove useless lcass method

* Compact rules

* be explicit

* Fix cli output test

* Fix typos

* denest functions and gitignore

* DAte no longer capital
2022-04-03 12:21:59 +01:00
Alan Cruickshank
d611a5d228 dbt 1.0.0 compatability (#2079)
* Introduce dbt 1.0.0 and update old references.

* Fix profiles dir import issues

* Fix a couple more errors

* Choose dbt_utils version based on the dbt version, change how we check if a model is disabled (#2087)

* Choose dbt_utils version based on the dbt version

* Skip the script if dbt templater is not installed

* In dbt 1.0, need to look up disabled models a different way

* Add fixture for changed whitespace in dbt_utils 0.8.0

* Database connection failures raise an exception in dbt 1.0.0

* Lint

* Some tweaks suggested in the PR review (#2088)

* Some tweaks suggested in the PR review

* Empty commit

Co-authored-by: Niall Woodward <niall@niallrees.com>
Co-authored-by: Barry Hart <barrywhart@yahoo.com>
2021-12-11 09:22:32 +00:00
Barry Hart
9dca7640c7 Improve Jinja whitespace handling in rules (#1647)
* Issue 1437: Robust Jinja raw/templated slice mapping

* Parametrize the test

* Add a basic (no whitespace control) test case

* Now correctly handling block_start whitespace stripping

* Add test case

* Enhance test case

* Handle variables with whitespace stripping, add a test for it

* Tidying, update test case

* Handle comments

* Add test for issue 1608

* Fix some things that were causing lint messages

* The test for this issue needs to run "fix"

* Add missing file, update .gitignore

* "no cover"

* Trigger build

* PR review, "no cover"

* Add two more test cases, basic_data and strip_right_data

* PR review

* More comments on the algorithm

* More comments

* Handle right whitespace stripping similarly as left

* Test updates

* Remove test case changes that were added in this PR and stopped working when I added handling for right whitespace stripping

* Add "pragma no cover"

* Change how I'm doing "pragma: no cover" to be simpler/safer

Co-authored-by: Barry Hart <barry.hart@mailchimp.com>
Co-authored-by: Barry Pollard <barry@tunetheweb.com>
2021-10-16 12:45:57 -04:00
Alan Cruickshank
6a3898d83a Rethink sequence_files in dbt templater. (#1563) 2021-10-08 22:58:21 +01:00
Barry Pollard
defb1749d5 Add .editorconfig config and enforce style rules (#1410)
* Add editorconfig, giattributes and convert some files

* Convert more files

* Update .gitatttributes

Co-authored-by: Barry Hart <barrywhart@yahoo.com>

Co-authored-by: Barry Hart <barrywhart@yahoo.com>
2021-09-15 19:02:08 +01:00
Alan Cruickshank
966877e57c Add updated sqlfluff graphics and source. (#1315)
* Add updated sqlfluff graphics and source.

* Apply tinypng

* SVG crop

* Add images readme

* Ignore DS_store files

Co-authored-by: Barry <barry@tunetheweb.com>
2021-08-26 10:09:07 +01:00
Greg Clunies
3b1ed4aabb Issue 861: Fix dbt templater removing trailing newline in source SQL (#868)
* Add newline to compiled_sql string

* Prevent unwanted deletion of trailing newline in source SQL when using dbt compiler

* Revert .gitignore to include conda env

* Update CHANGELOG

* Update CHANGELOG

* remove conda env, causes confusion

* remove agate requirement, let owners decide

* Update CHANGELOG

* Run black

* Add newline to .gitignore - next level irony

* Cleanup WET code

* Run black

* Use list comprehension

* Save progress

* Simplified approach to handle trailing newlines

* Run black

* add back in exclude_rules

* Remove appended newline in dbt_test. Handled by templaters/dbt.py directly

* Fix test__templater_dbt_templating_test_lex to reflect improved trailing newline handling

* Ignore flake8 D418 errors

* Remove unneeded #noqa after merging in linting fixes from upstream

* Update dbt tests

* Trt removign whitespace from test assert string

* Remove commented code used for debugging

* Update comments for clarity

* Update docs

* Run black

* Fix CHANGELOG typo

* Update .gitignore to account for direnv config

* Fix comment typo in templaters/dbt.py

* Remove unneeded comment

* Simplify tests using fixtures

* Run black

* #noqa fixes

* Fix fixture list

* Simplify docstring

Co-authored-by: NiallRees <niall@niallrees.com>
2021-04-13 12:17:52 +01:00
Alan Cruickshank
ab5ffc3c8f Mask linting errors around templated sections. (#745)
* mask linting errors around templated sections.

* typo

* Correct gitignore

Co-authored-by: NiallRees <niall@niallrees.com>
2021-02-11 21:55:21 +00:00
Bart Riepe
ea71d4f5fa Allow more complex offset structures (#702) 2021-01-22 16:41:47 +00:00
Barry Hart
b1835231c5 Add test generator to exercise lots of different mixtures of relational operators 2021-01-10 17:46:57 -05:00
NiallRees
e26a042aaf 🗃️ Split up rule tests into yml files (#553) 2020-11-21 21:18:32 -05:00
Alan
d939eeca86 Skip local python testing scripts 2020-11-02 13:43:48 +00:00
pwildenhain
b8b4375ece Improve code coverage for rules (#515)
* 🙈 Ignore coverage report

* 🤝 Implement common sense coverage exclusions
1. Don't need to test `sqlfluff/__main__ since it's tested by subprocess.check_output rather than imported directly

2. Ignore version checks for now, since we're not implementing any different behaviors by version. I could also exclude these lines in particular with a `pragma: no cover` if you prefer.

* ♻️ Simplify trailing whitespace fixes

This was interesting -- the `else:` was never being hit in the tests, which didn't originally make sense to me 🤔 because that should break the `while True:` condition. It must be getting broken by something else? Either way, this makes it more readable and ensures coverage

* 💀 Remove dead code from Rule_L002._eval

I tried to create test cases to hit this code, but couldn't figure it out. It seems like checking at the beginning and checking at newlines covers all use cases for this rule.

* 💪 Improve coverage for Rule_L003._make_indent

There didn't seem to be any test cases for indent_unit = tabs, so +1 for
that. Also got rid of the error handling, which is now taken care of at
the config level

* 💀 Remove dead code from Rule_L003._coerce_indent_to

🤔 Seems like the intention of this was to remove any hanging
indents, when an incorrect indent is found on a segement. Try as I may
I couldn't hit this line, so I'm guessing we don't need it
🤷‍♂️

* 💀 Remove dead code from Rule_L003._eval

1. `else: pass` isn't necessary
2. No need to check if a comment wasn't found, because previous if logic
   already ensures that a comment was found

*  Add L008 test case comma w/o whitespace

* 🐛 Allow capitalised keywords

This was actually a bug that we squashed early, since I discovered that
it wasn't being hit by tests, and then my test case failed.

Hooray for coverage!  🎉

Also get rid of config checking, which is handled separately

* 🔥 Remove uneeded check from Rule_L011

If there wasn't whitespace or a new line before an alias, then it
wouldn't register as an alias in SQL.

Unless you can think of a test case, then I think this is safe to remove

*  Test allow_scalar cases for Rule_L013

* 💀 Remove dead code from Rule_L016._eval_line_for_break.Section

There was some other untested code in here, but I thought it better to
leave it in, since it's specifically meant to catch cases we can't
imagine

* 💀 Remove dead code from Rule_L017._eval

* 💀 Remove dead code from Rule_L003._eval_line_for_breaks

I made sure that getting a double comma was impossible, don't think we
need to keep this check in here

* 💀 Remove dead code from Rule_L018._eval

*  Add test cases for Rule_L018

* 💀 Remove dead code from Rule_L019._last_code_seg

There is no need for this check, since we'll always check that
len(raw_stack) > 1 in self._eval

*  Add test for Rule_L020.__lint_references_and_aliases

*  Add test cases for Rule_L021._eval

*  Add test cases for Rule_L022._eval

* 📛 Rename var for easier debugging

In pdb, `r` shows the current stack, and so it made it hard to inspect
this value in debugging

* 🐛 Fix wildcard reference name

Score another for discovering unlinted lines 🎉

* ↩️ Revert f6d97ed, add test case instead"

* 🤦‍♂️ Add back in removed test
2020-10-30 11:20:16 -04:00
Alan Cruickshank
a480eff2d0 Mysql dialect, ALTER TABLE, Parameters 2020-05-11 11:42:02 +01:00
Azhar Dewji
9f0cf413c0 Fix diff-quality attempting to check deleted files 2020-05-07 10:45:54 -04:00
Nolan Conaway
6dd84b62de add JSON output support to the linter. 2020-02-25 20:12:29 -05:00
Nolan Conaway
24a33ea0b9 add fix stdin handler
cleaned up my git history for you alan sorry i am so terrible :-)
2019-12-14 21:07:07 -05:00
Alan Cruickshank
939a860972 More Documentation 2019-11-19 18:04:09 +00:00
Nolan Conaway
85ce1cef1f add venv to gitignore 2019-11-03 21:28:06 -05:00
Alan Cruickshank
566cdef9b8 Rationalise Config Files 2019-10-25 21:27:51 +01:00
Alan Cruickshank
8f2d4af77e move directory creating into individual channels 2018-11-26 10:37:08 +00:00
Alan Cruickshank
0fa773c80c A bit more circleci sugar to upload test reports 2018-11-15 12:18:17 +00:00
Alan Cruickshank
f82b5c38c7 More CircleCI config changes 2018-11-14 10:41:36 +00:00
Alan Cruickshank
df29939940 Tox changes for code coverage, travis and circleci 2018-11-14 09:32:27 +00:00
Alan Cruickshank
e18280cb8c more gitignore changes 2018-11-07 23:10:42 +00:00
Alan Cruickshank
9ed19f2ef7 update gitignore 2018-11-07 23:08:22 +00:00
Alan Cruickshank
f18f3941d6 More Gitignore 2018-11-04 14:49:41 +00:00
Alan Cruickshank
bfbbdc2706 Update gitignore 2018-11-04 13:59:42 +00:00
Alan Cruickshank
b04a0f118e Updating gitignore 2018-11-04 13:59:10 +00:00
Alan Cruickshank
dbba075fdd more gitignore 2018-10-30 20:55:30 +00:00
Alan Cruickshank
837f748d2b Starting Test Driven Dev 2018-10-30 20:55:10 +00:00