Unable to compile typescript scripts for crawler

[Inkvi]

Hello,

I am trying to compile typescript scripts for crawler using tsc following these commands

cd crawler
npm run build

It fails because it can't find node modules present under node_modules and some syntax errors. I am not familiar with typescript configuration but I assume some options might be missing crawler/tsconfig.json to make the compilation work out of the box.

➜  crawler git:(master) ✗ npm run build

> crawler@1.1.4 build /Users/a/dev/Crawling-Infrastructure/crawler
> rm -rf dist/ && tsc

../lib/misc/helpers.ts:4:18 - error TS2307: Cannot find module 'glob'.

4 import glob from 'glob';
                   ~~~~~~

../lib/misc/helpers.ts:5:27 - error TS2307: Cannot find module 'detect-newline'.

5 import detectNewline from 'detect-newline';
                            ~~~~~~~~~~~~~~~~

../lib/misc/logger.ts:7:30 - error TS2307: Cannot find module 'winston'.

7 import {format, Logger} from 'winston';
                               ~~~~~~~~~

../lib/misc/logger.ts:10:26 - error TS2307: Cannot find module 'winston'.

10 import * as winston from 'winston';
                            ~~~~~~~~~

../lib/misc/logger.ts:12:24 - error TS2307: Cannot find module 'winston'.

12 export { Logger } from 'winston';
                          ~~~~~~~~~

../lib/misc/logger.ts:63:46 - error TS2354: This syntax requires an imported helper but module 'tslib' cannot be found.

63     const myFormat = winston.format.printf(({level, message, label, timestamp}) => {
                                                ~~~~~

../lib/misc/logger.ts:63:46 - error TS7031: Binding element 'level' implicitly has an 'any' type.

63     const myFormat = winston.format.printf(({level, message, label, timestamp}) => {
                                                ~~~~~

../lib/misc/logger.ts:63:53 - error TS7031: Binding element 'message' implicitly has an 'any' type.

63     const myFormat = winston.format.printf(({level, message, label, timestamp}) => {
                                                       ~~~~~~~

../lib/misc/logger.ts:63:62 - error TS7031: Binding element 'label' implicitly has an 'any' type.

63     const myFormat = winston.format.printf(({level, message, label, timestamp}) => {
                                                                ~~~~~

../lib/misc/logger.ts:63:69 - error TS7031: Binding element 'timestamp' implicitly has an 'any' type.

63     const myFormat = winston.format.printf(({level, message, label, timestamp}) => {
                                                                       ~~~~~~~~~

../lib/storage/storage.ts:4:17 - error TS2307: Cannot find module 'aws-sdk'.

4 import AWS from 'aws-sdk'
                  ~~~~~~~~~

../lib/storage/storage.ts:45:39 - error TS7006: Parameter 'error' implicitly has an 'any' type.

45       this.client.putObject(request, (error, data) => {
                                         ~~~~~

../lib/storage/storage.ts:45:46 - error TS7006: Parameter 'data' implicitly has an 'any' type.

45       this.client.putObject(request, (error, data) => {
                                                ~~~~

../lib/storage/storage.ts:104:28 - error TS7006: Parameter 'error' implicitly has an 'any' type.

104         request, function (error, data) {
                               ~~~~~

../lib/storage/storage.ts:104:35 - error TS7006: Parameter 'data' implicitly has an 'any' type.

104         request, function (error, data) {
                                      ~~~~

../lib/storage/storage.ts:286:51 - error TS7006: Parameter 'err' implicitly has an 'any' type.

286       this.client.listObjectsV2(params, function (err, data) {
                                                      ~~~

../lib/storage/storage.ts:286:56 - error TS7006: Parameter 'data' implicitly has an 'any' type.

286       this.client.listObjectsV2(params, function (err, data) {
                                                           ~~~~

../lib/storage/storage.ts:303:49 - error TS7006: Parameter 'err' implicitly has an 'any' type.

303       this.client.listObjects(params, function (err, data) {
                                                    ~~~

../lib/storage/storage.ts:303:54 - error TS7006: Parameter 'data' implicitly has an 'any' type.

303       this.client.listObjects(params, function (err, data) {
                                                         ~~~~

../lib/storage/storage.ts:321:41 - error TS7006: Parameter 'err' implicitly has an 'any' type.

321       this.client.deleteObject(params, (err, data) => {
                                            ~~~

../lib/storage/storage.ts:321:46 - error TS7006: Parameter 'data' implicitly has an 'any' type.

321       this.client.deleteObject(params, (err, data) => {
                                                 ~~~~


Found 21 errors.

npm ERR! code ELIFECYCLE
npm ERR! errno 2
npm ERR! crawler@1.1.4 build: `rm -rf dist/ && tsc`
npm ERR! Exit status 2
npm ERR!
npm ERR! Failed at the crawler@1.1.4 build script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR!     /Users/a/.npm/_logs/2020-05-16T00_40_16_459Z-debug.log```

[NikolaiT]

Should be fixed by now. Can you confirm?

[Inkvi]

The original errors disappeared but a new one came up

➜  crawler git:(master) npm run build

> crawler@1.1.5 build /Users/username/dev/Crawling-Infrastructure/crawler
> rm -rf dist/ && tsc

error TS2688: Cannot find type definition file for 'node'.


Found 1 error.

npm ERR! code ELIFECYCLE
npm ERR! errno 2
npm ERR! crawler@1.1.5 build: `rm -rf dist/ && tsc`
npm ERR! Exit status 2
npm ERR! 

[Aditya94A]

@Inkvi @NikolaiT Were you able to fix this?

I'm getting the following error:

node_modules/@types/mongoose/index.d.ts:3855:13 - error TS2320: Interface 'CollectionBase' cannot simultaneously extend types 'Col
lection<any>' and 'Collection<any>'.
  Named property 'update' of types 'Collection<any>' and 'Collection<any>' are not identical.

3855   interface CollectionBase extends mongodb.Collection {
                 ~~~~~~~~~~~~~~

node_modules/@types/mongoose/index.d.ts:3855:13 - error TS2320: Interface 'CollectionBase' cannot simultaneously extend types 'Col
lection<any>' and 'Collection<any>'.
  Named property 'updateMany' of types 'Collection<any>' and 'Collection<any>' are not identical.

3855   interface CollectionBase extends mongodb.Collection {
                 ~~~~~~~~~~~~~~

node_modules/@types/mongoose/index.d.ts:3855:13 - error TS2320: Interface 'CollectionBase' cannot simultaneously extend types 'Col
lection<any>' and 'Collection<any>'.
  Named property 'updateOne' of types 'Collection<any>' and 'Collection<any>' are not identical.

3855   interface CollectionBase extends mongodb.Collection {
                 ~~~~~~~~~~~~~~

node_modules/@types/mongoose/index.d.ts:3855:13 - error TS2320: Interface 'CollectionBase' cannot simultaneously extend types 'Col
lection<any>' and 'Collection<any>'.
  Named property 'watch' of types 'Collection<any>' and 'Collection<any>' are not identical.

3855   interface CollectionBase extends mongodb.Collection {
                 ~~~~~~~~~~~~~~


Found 60 errors.

[Adriman2]

@Inkvi @NikolaiT Were you able to fix this?

I'm getting the following error:

node_modules/@types/mongoose/index.d.ts:3855:13 - error TS2320: Interface 'CollectionBase' cannot simultaneously extend types 'Col
lection<any>' and 'Collection<any>'.
  Named property 'update' of types 'Collection<any>' and 'Collection<any>' are not identical.

3855   interface CollectionBase extends mongodb.Collection {
                 ~~~~~~~~~~~~~~

node_modules/@types/mongoose/index.d.ts:3855:13 - error TS2320: Interface 'CollectionBase' cannot simultaneously extend types 'Col
lection<any>' and 'Collection<any>'.
  Named property 'updateMany' of types 'Collection<any>' and 'Collection<any>' are not identical.

3855   interface CollectionBase extends mongodb.Collection {
                 ~~~~~~~~~~~~~~

node_modules/@types/mongoose/index.d.ts:3855:13 - error TS2320: Interface 'CollectionBase' cannot simultaneously extend types 'Col
lection<any>' and 'Collection<any>'.
  Named property 'updateOne' of types 'Collection<any>' and 'Collection<any>' are not identical.

3855   interface CollectionBase extends mongodb.Collection {
                 ~~~~~~~~~~~~~~

node_modules/@types/mongoose/index.d.ts:3855:13 - error TS2320: Interface 'CollectionBase' cannot simultaneously extend types 'Col
lection<any>' and 'Collection<any>'.
  Named property 'watch' of types 'Collection<any>' and 'Collection<any>' are not identical.

3855   interface CollectionBase extends mongodb.Collection {
                 ~~~~~~~~~~~~~~


Found 60 errors.

I had the same error. Quick fixed it by changing the name of the mongoose directory.
Looks like this:

mv ../lib/node_modules/@types/mongoose ../lib/node_modules/@types/mongoose-deact

you can also try removing the directory

[thomas-z8]

@Adriman2
Thank you for the quick fix.
Indeed renaming ../lib/node_modules/@types/mongoose let the tsc compilation finish without error.
Yet when i ran the next command node dist/master/src/server.js it gave me the following error (not sure yet if it is related to the renaming or not):

Connecting to mongodb: undefined
Could not connect to undefined: MongooseError: The `uri` parameter to `openUri()` must be a string, got "undefined". Make sure the first parameter to `mongoose.connect()` or `mongoose.createConnection()` is a string.