Skip to content

Commit 80966b3

Browse files
tonyfasttonyfast
authored
Remove opinions and introduce a test loader. (ipython#39)
* Refactor the compile notebook and expose an export function to format our development code. * Add docstring information to the readme * Add a docstring test. * Explicit names for the compiler and exporter classes giving a nicer graph structure * End the markdown quotes on a new lines * Explicit class names for the parameterize functions. * Add an ability to load python files as modules with from_filename * Format source code. * Doctest a module in the tests. * Remove the tox requirements and use pytest syntax for the tests. * Add more tests * Skip the docstring test in 3.7 until the docstring is fixed * modify how the changelog is read and remove the markdown version * Include the changelog.ipynb in the manifest * Make Notebook classes callable and make from_filename an regular method. * Use the default IPython transformer * Update the benchmark with the new from file code
1 parent 585af20 commit 80966b3

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

45 files changed

+1606
-974
lines changed

‎MANIFEST.in

+1-1
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,2 @@
1-
include LICENSE readme.md changelog.md
1+
include LICENSE readme.md changelog.ipynb src/notebooks/block_string.tpl
22
recursive-include src/importnb *.ipynb

‎benchmarks/benchmark.ipynb

+1-1
Original file line numberDiff line numberDiff line change
@@ -114,7 +114,7 @@
114114
"if repo and __name__ != '__main__':\n",
115115
" modules = {}\n",
116116
" for path in notebooks.glob('*.ipynb'):\n",
117-
" modules[path] = module = Timed.from_filename(path, stdout=True, stderr=True, display=display)\n",
117+
" modules[path] = module = Timed(stdout=True, stderr=True, display=display).from_filename(path)\n",
118118
" print(f\"{'🏆😩'[bool(module.__exception__)]} {path}\")\n",
119119
" "
120120
]

‎benchmarks/readme.ipynb

+5-14
Large diffs are not rendered by default.

‎benchmarks/readme.md

+4-9
Original file line numberDiff line numberDiff line change
@@ -55,15 +55,15 @@ Benchmarks for `importnb` explore the ability to import existing notebooks as mo
5555
🏆 ThinkDSP/code/chap09soln.ipynb
5656
😩 ThinkDSP/code/chap10.ipynb
5757
🏆 ThinkDSP/code/chap10preview.ipynb
58-
🏆 ThinkDSP/code/chap10soln.ipynb
58+
😩 ThinkDSP/code/chap10soln.ipynb
5959
😩 ThinkDSP/code/chap11.ipynb
6060
😩 ThinkDSP/code/chap11soln.ipynb
6161
😩 ThinkDSP/code/diff_int.ipynb
6262
🏆 ThinkDSP/code/phase.ipynb
6363
🏆 ThinkDSP/code/saxophone.ipynb
6464
🏆 ThinkDSP/code/scipy2015_demo.ipynb
6565
😩 ThinkDSP/code/voss.ipynb
66-
70.0% of the ThinkDSP notebooks loaded with a import threshold of 10.000000 seconds
66+
66.7% of the ThinkDSP notebooks loaded with a import threshold of 10.000000 seconds
6767
{'repo': 'https://www.github.com/jakevdp/PythonDataScienceHandbook', 'notebooks': 'PythonDataScienceHandbook/notebooks/'}
6868
==========
6969
🏆 PythonDataScienceHandbook/notebooks/00.00-Preface.ipynb
@@ -257,7 +257,7 @@ Benchmarks for `importnb` explore the ability to import existing notebooks as mo
257257

258258

259259
😩 PythonDataScienceHandbook/notebooks/05.07-Support-Vector-Machines.ipynb
260-
🏆 PythonDataScienceHandbook/notebooks/05.08-Random-Forests.ipynb
260+
😩 PythonDataScienceHandbook/notebooks/05.08-Random-Forests.ipynb
261261
🏆 PythonDataScienceHandbook/notebooks/05.09-Principal-Component-Analysis.ipynb
262262
😩 PythonDataScienceHandbook/notebooks/05.10-Manifold-Learning.ipynb
263263
😩 PythonDataScienceHandbook/notebooks/05.11-K-Means.ipynb
@@ -267,19 +267,14 @@ Benchmarks for `importnb` explore the ability to import existing notebooks as mo
267267
🏆 PythonDataScienceHandbook/notebooks/05.15-Learning-More.ipynb
268268
🏆 PythonDataScienceHandbook/notebooks/06.00-Figure-Code.ipynb
269269
🏆 PythonDataScienceHandbook/notebooks/Index.ipynb
270-
60.3% of the PythonDataScienceHandbook notebooks loaded with a import threshold of 10.000000 seconds
270+
58.8% of the PythonDataScienceHandbook notebooks loaded with a import threshold of 10.000000 seconds
271271

272272

273273

274274
![png](readme_files/readme_4_3.png)
275275

276276

277277

278-
```python
279-
280-
```
281-
282-
283278
```python
284279
!jupyter nbconvert --to markdown readme.ipynb
285280
```

‎changelog.ipynb

+18
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,24 @@
77
"## CHANGELOG"
88
]
99
},
10+
{
11+
"cell_type": "markdown",
12+
"metadata": {},
13+
"source": [
14+
"# 0.3.0\n",
15+
"\n",
16+
"* Include `Partial`, `Lazy`, and `NotebookTest` loaders.\n",
17+
"* Transform markdown cells to literate block strings so they are included in the ast.\n",
18+
" * `__doc__`'s are extracted from the first markdown cell or normal source code from a code cell.\n",
19+
"* Export the python source code with `black`.\n",
20+
"* `Notebook.from_filename` is a loader for paths and strings.\n",
21+
"* Add `importnb.nbtest` for notebook testing tools..\n",
22+
"* Benchmark `importnb` against existing notebooks.\n",
23+
"* Include a `watchdog` trick to watch tests..\n",
24+
"* Extend the project to >= 3.4\n",
25+
"* Use nbviewer/github hierachy for the docs."
26+
]
27+
},
1028
{
1129
"cell_type": "markdown",
1230
"metadata": {},

‎changelog.md

-37
This file was deleted.

‎readme.ipynb

+63-51
Original file line numberDiff line numberDiff line change
@@ -12,7 +12,11 @@
1212
"\n",
1313
"\n",
1414
"\n",
15-
" pip install importnb"
15+
" pip install importnb\n",
16+
" \n",
17+
"---\n",
18+
"\n",
19+
" conda install -c conda-forge importnb"
1620
]
1721
},
1822
{
@@ -30,7 +34,7 @@
3034
"source": [
3135
"### For brevity\n",
3236
"\n",
33-
"[`importnb.loader`](src/notebooks/loader.ipynb) will find notebooks avaiable anywhere along the [`sys.path`](https://docs.python.org/2/library/sys.html#sys.path)."
37+
"[`importnb.loader`](src/notebooks/loader.ipynb) will find notebooks available anywhere along the [`sys.path`](https://docs.python.org/2/library/sys.html#sys.path)."
3438
]
3539
},
3640
{
@@ -151,6 +155,33 @@
151155
" import readme"
152156
]
153157
},
158+
{
159+
"cell_type": "markdown",
160+
"metadata": {},
161+
"source": [
162+
"### Docstring\n",
163+
"\n",
164+
"The first cell is the module docstring."
165+
]
166+
},
167+
{
168+
"cell_type": "code",
169+
"execution_count": 8,
170+
"metadata": {},
171+
"outputs": [
172+
{
173+
"name": "stdout",
174+
"output_type": "stream",
175+
"text": [
176+
"__importnb__ imports notebooks as modules & packages.\n"
177+
]
178+
}
179+
],
180+
"source": [
181+
" if __name__ == '__main__':\n",
182+
" print(readme.__doc__.splitlines()[0])"
183+
]
184+
},
154185
{
155186
"cell_type": "markdown",
156187
"metadata": {},
@@ -179,7 +210,7 @@
179210
},
180211
{
181212
"cell_type": "code",
182-
"execution_count": 8,
213+
"execution_count": 9,
183214
"metadata": {},
184215
"outputs": [],
185216
"source": [
@@ -197,7 +228,7 @@
197228
},
198229
{
199230
"cell_type": "code",
200-
"execution_count": 9,
231+
"execution_count": 10,
201232
"metadata": {},
202233
"outputs": [
203234
{
@@ -206,7 +237,7 @@
206237
"<Signature (*, foo=42)>"
207238
]
208239
},
209-
"execution_count": 9,
240+
"execution_count": 10,
210241
"metadata": {},
211242
"output_type": "execute_result"
212243
}
@@ -333,45 +364,14 @@
333364
},
334365
{
335366
"cell_type": "code",
336-
"execution_count": 10,
367+
"execution_count": 11,
337368
"metadata": {},
338369
"outputs": [
339-
{
340-
"name": "stdout",
341-
"output_type": "stream",
342-
"text": [
343-
"src/notebooks/capture.ipynb\n",
344-
"src/notebooks/compiler_ipython.ipynb\n",
345-
"src/notebooks/compiler_python.ipynb\n",
346-
"src/notebooks/decoder.ipynb\n",
347-
"src/notebooks/exporter.ipynb\n",
348-
"src/notebooks/loader.ipynb\n",
349-
"src/notebooks/parameterize.ipynb\n",
350-
"src/notebooks/utils/__init__.ipynb\n",
351-
"src/notebooks/utils/ipython.ipynb\n"
352-
]
353-
},
354-
{
355-
"name": "stderr",
356-
"output_type": "stream",
357-
"text": [
358-
"test_import (src.importnb.tests.test_unittests.TestContext) ... "
359-
]
360-
},
361-
{
362-
"name": "stdout",
363-
"output_type": "stream",
364-
"text": [
365-
"src/notebooks/utils/pytest_plugin.ipynb\n",
366-
"src/notebooks/utils/setup.ipynb\n",
367-
"src/notebooks/utils/watch.ipynb\n"
368-
]
369-
},
370370
{
371371
"name": "stderr",
372372
"output_type": "stream",
373373
"text": [
374-
"ok\n",
374+
"test_import (src.importnb.tests.test_unittests.TestContext) ... ok\n",
375375
"test_reload_with_context (src.importnb.tests.test_unittests.TestContext) ... ok\n",
376376
"test_failure (src.importnb.tests.test_unittests.TestExtension) ... expected failure\n",
377377
"test_import (src.importnb.tests.test_unittests.TestExtension) ... ok\n",
@@ -380,7 +380,7 @@
380380
"test_imports (src.importnb.tests.test_unittests.TestRemote) ... skipped 'requires IP'\n",
381381
"\n",
382382
"----------------------------------------------------------------------\n",
383-
"Ran 7 tests in 2.023s\n",
383+
"Ran 7 tests in 2.021s\n",
384384
"\n",
385385
"OK (skipped=1, expected failures=1)\n"
386386
]
@@ -389,38 +389,50 @@
389389
"source": [
390390
" if __name__ == '__main__':\n",
391391
" from pathlib import Path\n",
392-
" from importnb.compiler_python import ScriptExporter\n",
392+
" from nbconvert.exporters.python import PythonExporter\n",
393+
" from importnb.compile import export\n",
393394
" for path in Path('src/notebooks/').rglob(\"\"\"*.ipynb\"\"\"): \n",
394395
" if 'checkpoint' not in str(path):\n",
395-
" src = ScriptExporter().from_filename(path)[0]\n",
396-
" try:\n",
397-
" import black\n",
398-
" src = black.format_str(src, 100)\n",
399-
" except: ...\n",
400-
"\n",
401-
" print(path)\n",
402-
" (Path('src/importnb') / path.with_suffix('.py').relative_to('src/notebooks')).write_text(src)\n",
396+
" export(path, Path('src/importnb') / path.with_suffix('.py').relative_to('src/notebooks'))\n",
403397
" \n",
404398
" __import__('unittest').main(module='src.importnb.tests.test_unittests', argv=\"discover --verbose\".split(), exit=False) \n"
405399
]
406400
},
407401
{
408402
"cell_type": "code",
409-
"execution_count": 11,
403+
"execution_count": 12,
410404
"metadata": {},
411405
"outputs": [
412406
{
413407
"name": "stdout",
414408
"output_type": "stream",
415409
"text": [
416410
"[NbConvertApp] Converting notebook readme.ipynb to markdown\n",
417-
"[NbConvertApp] Writing 7132 bytes to readme.md\n"
411+
"[NbConvertApp] Writing 7121 bytes to readme.md\n"
418412
]
419413
}
420414
],
421415
"source": [
422-
" !jupyter nbconvert --to markdown readme.ipynb"
416+
" if __name__ == '__main__':\n",
417+
" !jupyter nbconvert --to markdown readme.ipynb"
423418
]
419+
},
420+
{
421+
"cell_type": "markdown",
422+
"metadata": {},
423+
"source": [
424+
" if __name__ == '__main__':\n",
425+
" from IPython.display import display, Image\n",
426+
" !pyreverse importnb -opng -pimportnb\n",
427+
" display(*map(Image, ('classes_importnb.png', 'packages_importnb.png')))"
428+
]
429+
},
430+
{
431+
"cell_type": "code",
432+
"execution_count": null,
433+
"metadata": {},
434+
"outputs": [],
435+
"source": []
424436
}
425437
],
426438
"metadata": {

0 commit comments

Comments
 (0)