Added l2tdevtools configuration files and generated dependency files #790

joachimmetz · 2019-01-17T20:58:53Z

@berggren do not merge this yet, but please check if you're OK with these changes

Depends on #832

berggren · 2019-01-22T08:13:13Z

Thanks for putting this together! It looks good, but I need to test this PR. I let you know.

kiddinn

Adding few comments to this PR

setup.cfg

config/linux/gift_ppa_install.sh

dependencies.ini

requirements.txt

joachimmetz · 2019-02-09T06:54:17Z

@kiddinn FYI changing this in dependencies.ini and running l2tdevtools update-dependencies.py will re-generate the files you flagged.

joachimmetz · 2019-02-09T06:55:26Z

Also pandas seem to have been added

diff --git a/requirements.txt b/requirements.txt
index e594bf5..ccb1d33 100644
--- a/requirements.txt
+++ b/requirements.txt
@@ -48,17 +48,16 @@ mccabe==0.6.1             # via pylint
 mock==2.0.0
 neo4jrestclient==2.1.1
 nose==1.3.7
-numpy==1.13.3             # via datasketch
-pandas>=0.23
+numpy==1.13.3             # via datasketch, pandas
+pandas==0.24.1
 parameterized==0.6.1
 pbr==3.1.1                # via mock
 pycparser==2.18           # via cffi
-pycypher==0.5.9
 pyjwt==1.6.4
 pylint==1.7.2
 python-dateutil==2.6.1
 python-editor==1.0.3      # via alembic
-pytz==2017.2              # via celery, flask-restful
+pytz==2017.2              # via celery, flask-restful, pandas
 pyyaml==4.2b4
 redis==2.10.6
 requests==2.20.1

joachimmetz · 2019-02-09T07:25:04Z

@berggren @kiddinn I see that pandas is used in api_client/python/timesketch_api_client/client.py but this is not used in the timesketch module?

joachimmetz · 2019-02-09T08:52:06Z

sudo add-apt-repository universe
sudo add-apt-repository ppa:gift/stable
./config/linux/gift_ppa_install.sh

kiddinn · 2019-02-09T08:54:26Z

@berggren @kiddinn I see that pandas is used in api_client/python/timesketch_api_client/client.py but this is not used in the timesketch module?

@joachimmetz pandas is also used in the analyzers interface and in the browser timeframe analyzer

kiddinn

few comments

kiddinn · 2019-02-09T08:55:23Z

.travis.yml

-  - pip install .
+  - pip install -r requirements.txt
+  - pip install -r test_requirements.txt
+  - pip install pandas


why have this here, and not in requirements.txt?

@kiddinn this PR is WIP, I've not added a reviewer yet. The reason why this is here is because of the previous question about pandas.

dependencies.ini

kiddinn · 2019-02-09T08:56:27Z

timesketch.ini

+name: timesketch
+name_description: Timesketch
+maintainer: Timesketch development team <timesketch-dev@googlegroups.com>
+homepage_url: https://github.com/log2timeline/dfdatetime


this is not the correct homepage url

joachimmetz · 2019-02-09T09:19:11Z

@kiddinn What is the minimum required version of pandas?

kiddinn · 2019-02-09T09:25:01Z

@kiddinn What is the minimum required version of pandas?

0.24 is the optimal but offline discussions on how far we can go back... that is whether we want to support xenial (Ubuntu), which only has 0.17

joachimmetz · 2019-02-10T12:10:09Z

Xenial support is going to involve updating several more dependencies log2timeline/l2tdevtools#447 so IMHO not a priority.

joachimmetz · 2019-03-19T20:45:57Z

New dependencies:

altair (not provided in bionic)
- added to GIFT
attrs which is named python-attr
configparser 3.5.0
entrypoints 0.2.3
pyrsistent (not provided in bionic)
- added to GIFT ~~Fix failing pyrsistent build on GIFT PPA testing log2timeline/l2tdevtools#636~~
toolz 0.8.2
jsonschema 2.6.0
typing (not provided in bionic)

Pre 3.6 alternative for typing is https://pypi.org/project/python_version/

joachimmetz · 2019-03-21T08:47:11Z

~~TODO: address log2timeline/l2tdevtools#642~~ not blocking

joachimmetz · 2019-03-21T09:24:37Z

Python 2 requires python-typing https://packages.ubuntu.com/search?keywords=python-typing

joachimmetz · 2019-03-21T19:53:43Z

After updating the pip tests fail, also some warnings in the pip output:

celery 4.2.2 has requirement billiard<3.6.0,>=3.5.0.2, but you'll have billiard 3.6.0.0 which is incompatible.
celery 4.2.2 has requirement kombu<4.4,>=4.2.0, but you'll have kombu 4.4.0 which is incompatible.

joachimmetz · 2019-03-24T16:13:15Z

@berggren @kiddinn this PR is ready for review now.

I've reverted using the auto generated requirements.txt due to #854

joachimmetz · 2019-04-02T05:07:19Z

@berggren @kiddinn friendly ping

berggren · 2019-04-02T10:18:45Z

@joachimmetz Ack, taking a look today.

kiddinn

this LGTM, but I'll allow johan to make the final green light

berggren

LGTM, when thew conflicts (travis/install.sh) is resolved I will merge this.

joachimmetz · 2019-04-02T20:26:10Z

@berggren merge conflict resolved

berggren

LGTM

kiddinn reviewed Jan 30, 2019

View reviewed changes

setup.cfg Outdated Show resolved Hide resolved

config/linux/gift_ppa_install.sh Outdated Show resolved Hide resolved

dependencies.ini Outdated Show resolved Hide resolved

requirements.txt Outdated Show resolved Hide resolved

joachimmetz force-pushed the l2tdevtools branch 2 times, most recently from 3666922 to 79c2c2b Compare February 9, 2019 07:36

kiddinn reviewed Feb 9, 2019

View reviewed changes

joachimmetz force-pushed the l2tdevtools branch from 1869694 to f02724f Compare February 9, 2019 09:18

joachimmetz force-pushed the l2tdevtools branch 2 times, most recently from 67104b7 to 2e381ec Compare February 9, 2019 19:21

joachimmetz force-pushed the l2tdevtools branch 4 times, most recently from 349db06 to c9d48e5 Compare March 21, 2019 06:23

joachimmetz force-pushed the l2tdevtools branch from c9d48e5 to fa51e42 Compare March 21, 2019 09:16

joachimmetz force-pushed the l2tdevtools branch from 73ec70f to 4bf2cad Compare March 21, 2019 19:22

joachimmetz added the Blocked The issue or PR is blocked on another issue label Mar 21, 2019

joachimmetz force-pushed the l2tdevtools branch 2 times, most recently from aadd39d to 2d1cd64 Compare March 21, 2019 19:34

joachimmetz mentioned this pull request Mar 22, 2019

auto-generated requirements.txt file causes pip tests to fail #854

Closed

joachimmetz force-pushed the l2tdevtools branch from 4fb9059 to 738bd62 Compare March 24, 2019 15:47

joachimmetz removed the Blocked The issue or PR is blocked on another issue label Mar 24, 2019

joachimmetz requested review from berggren and kiddinn March 24, 2019 16:12

joachimmetz assigned berggren Mar 24, 2019

joachimmetz force-pushed the l2tdevtools branch from 738bd62 to 7a5af16 Compare April 1, 2019 18:54

joachimmetz force-pushed the l2tdevtools branch from 7a5af16 to 35bbfd7 Compare April 2, 2019 09:39

kiddinn reviewed Apr 2, 2019

View reviewed changes

berggren previously approved these changes Apr 2, 2019

View reviewed changes

Added l2tdevtools configuration files and generated dependency files

6cfdac8

joachimmetz dismissed berggren’s stale review via 6cfdac8 April 2, 2019 20:25

joachimmetz force-pushed the l2tdevtools branch from 35bbfd7 to 6cfdac8 Compare April 2, 2019 20:25

joachimmetz requested review from kiddinn and berggren April 2, 2019 20:25

berggren approved these changes Apr 3, 2019

View reviewed changes

berggren merged commit 44ee27b into google:master Apr 3, 2019

joachimmetz deleted the l2tdevtools branch April 3, 2019 21:29

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Added l2tdevtools configuration files and generated dependency files #790

Added l2tdevtools configuration files and generated dependency files #790

joachimmetz commented Jan 17, 2019 •

edited

Loading

berggren commented Jan 22, 2019

kiddinn left a comment

joachimmetz commented Feb 9, 2019

joachimmetz commented Feb 9, 2019 •

edited

Loading

joachimmetz commented Feb 9, 2019

joachimmetz commented Feb 9, 2019

kiddinn commented Feb 9, 2019

kiddinn left a comment

kiddinn Feb 9, 2019

joachimmetz Feb 9, 2019 •

edited

Loading

kiddinn Feb 9, 2019

joachimmetz Feb 9, 2019

joachimmetz commented Feb 9, 2019

kiddinn commented Feb 9, 2019 •

edited

Loading

joachimmetz commented Feb 10, 2019

joachimmetz commented Mar 19, 2019 •

edited

Loading

joachimmetz commented Mar 21, 2019 •

edited

Loading

joachimmetz commented Mar 21, 2019

joachimmetz commented Mar 21, 2019

joachimmetz commented Mar 24, 2019 •

edited

Loading

joachimmetz commented Apr 2, 2019

berggren commented Apr 2, 2019

kiddinn left a comment

berggren left a comment

joachimmetz commented Apr 2, 2019

berggren left a comment

Added l2tdevtools configuration files and generated dependency files #790

Added l2tdevtools configuration files and generated dependency files #790

Conversation

joachimmetz commented Jan 17, 2019 • edited Loading

berggren commented Jan 22, 2019

kiddinn left a comment

Choose a reason for hiding this comment

joachimmetz commented Feb 9, 2019

joachimmetz commented Feb 9, 2019 • edited Loading

joachimmetz commented Feb 9, 2019

joachimmetz commented Feb 9, 2019

kiddinn commented Feb 9, 2019

kiddinn left a comment

Choose a reason for hiding this comment

kiddinn Feb 9, 2019

Choose a reason for hiding this comment

joachimmetz Feb 9, 2019 • edited Loading

Choose a reason for hiding this comment

kiddinn Feb 9, 2019

Choose a reason for hiding this comment

joachimmetz Feb 9, 2019

Choose a reason for hiding this comment

joachimmetz commented Feb 9, 2019

kiddinn commented Feb 9, 2019 • edited Loading

joachimmetz commented Feb 10, 2019

joachimmetz commented Mar 19, 2019 • edited Loading

joachimmetz commented Mar 21, 2019 • edited Loading

joachimmetz commented Mar 21, 2019

joachimmetz commented Mar 21, 2019

joachimmetz commented Mar 24, 2019 • edited Loading

joachimmetz commented Apr 2, 2019

berggren commented Apr 2, 2019

kiddinn left a comment

Choose a reason for hiding this comment

berggren left a comment

Choose a reason for hiding this comment

joachimmetz commented Apr 2, 2019

berggren left a comment

Choose a reason for hiding this comment

joachimmetz commented Jan 17, 2019 •

edited

Loading

joachimmetz commented Feb 9, 2019 •

edited

Loading

joachimmetz Feb 9, 2019 •

edited

Loading

kiddinn commented Feb 9, 2019 •

edited

Loading

joachimmetz commented Mar 19, 2019 •

edited

Loading

joachimmetz commented Mar 21, 2019 •

edited

Loading

joachimmetz commented Mar 24, 2019 •

edited

Loading