Skip to content

Commit

Permalink
check for empty model files (#28)
Browse files Browse the repository at this point in the history
* check for empty model files
* check for missing tags property
  • Loading branch information
owlas authored Oct 7, 2021
1 parent eeef5c6 commit f1ec134
Show file tree
Hide file tree
Showing 4 changed files with 32 additions and 12 deletions.
7 changes: 7 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,13 @@

Recent and upcoming changes to dbt2looker

## 0.9.1
### Fixed
- Fixed bug where dbt2looker would crash if a dbt project contained an empty model

### Changed
- When filtering models by tag, models that have no tag property will be ignored

## 0.9.0
### Added
- Support for spark adapter (@chaimt)
Expand Down
29 changes: 21 additions & 8 deletions dbt2looker/parser.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
import json
import jsonschema
import importlib.resources
from typing import Dict, Optional, List
from typing import Dict, Optional, List, Union
from functools import reduce

from . import models
Expand Down Expand Up @@ -46,20 +46,33 @@ def parse_adapter_type(raw_manifest: dict):
return manifest.metadata.adapter_type


def tags_match(query_tag: str, model: models.DbtModel) -> bool:
try:
return query_tag in model.tags
except AttributeError:
return False
except ValueError:
# Is the tag just a string?
return query_tag == model.tags


def parse_models(raw_manifest: dict, tag=None) -> List[models.DbtModel]:
manifest = models.DbtManifest(**raw_manifest)
all_models: List[models.DbtModel] = [
node
for node in manifest.nodes.values()
if node.resource_type == 'model'
]
filtered_models = (
all_models if tag is None else [
model for model in all_models
if tag in model.tags
]
)
return filtered_models

# Empty model files have many missing parameters
for model in all_models:
if not hasattr(model, 'name'):
logging.error('Cannot parse model with id: "%s" - is the model file empty?', model.unique_id)
raise SystemExit('Failed')

if tag is None:
return all_models
return [model for model in all_models if tags_match(tag, model)]


def check_models_for_missing_column_types(dbt_typed_models: List[models.DbtModel]):
Expand Down
6 changes: 3 additions & 3 deletions poetry.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[tool.poetry]
name = "dbt2looker"
version = "0.9.0"
version = "0.9.1"
description = "Generate lookml view files from dbt models"
authors = ["oliverlaslett <oliver@gethubble.io>"]
license = "MIT"
Expand Down

0 comments on commit f1ec134

Please sign in to comment.