NodeJs custom module can't be required on Mac OS but working on windows

[XiamiYoung]

Description

I have a custom module need be shared by multiple lambdas from layer, when I add this custom module into dependencies and call sam local invoke, on windows it's working, on Mac OS it's failing. I've tried on Windows deploy this to AWS it's working so assuming it's a sam problem.

Steps to reproduce

1. Create a lambda by sam init --runtime nodejs -n testlayer -d npm
2. Create a custom module named my-module under testlayer

index.js

module.exports = {
    printMsg:function(){
        console.log("this is my module");
    }
}

package.json
{
  "name": "my-module",
  "version": "1.0.0",
  "description": "my-module",
  "main": "index.js",
  "scripts": {
    "test": ""
  },
  "author": "aaa",
  "license": "ISC",
  "dependencies": {

  }
}

3. Create a dependencies folder, create a nodejs folder under it, create a package.json

{
  "dependencies": {
    "my-module": "file:../../my-module"
  }
}

4. npm install
5. verify node_module folder, my-module is installed(as a folder link)
6. edit template.yaml to add layer

...
 myModuleLayer:
       Type: AWS::Serverless::LayerVersion
       Properties:
           LayerName: myModuleLayer
           Description: myModuleLayer
           ContentUri: dependencies/
           CompatibleRuntimes:
             - nodejs10.x
           LicenseInfo: 'MIT'
           RetentionPolicy: Retain
     .....
     then ref it from lambda function
     .....
     Layers:
     - !Ref myModuleLayer
     .....

7. edit my-module in app.js

var mymodule = require("my-module");

exports.lambdaHandler = async (event, context) => {
    mymodule.printMsg();
    try {
        // const ret = await axios(url);
        response = {
            'statusCode': 200,
            'body': JSON.stringify({
                message: 'hello world',
                // location: ret.data.trim()
            })
        }
    } catch (err) {
        console.log(err);
        return err;
    }
    return response
};

8. call sam local invoke --no-event, and check result

2019-10-25T08:43:01.666Z	undefined	ERROR	Uncaught Exception	{"errorType":"Runtime.ImportModuleError","errorMessage":"Error: Cannot find module 'my-module'","stack":["Runtime.ImportModuleError: Error: Cannot find module 'my-module'","    at _loadUserApp (/var/runtime/UserFunction.js:100:13)","    at Object.module.exports.load (/var/runtime/UserFunction.js:140:17)","    at Object.<anonymous> (/var/runtime/index.js:45:30)","    at Module._compile (internal/modules/cjs/loader.js:778:30)","    at Object.Module._extensions..js (internal/modules/cjs/loader.js:789:10)","    at Module.load (internal/modules/cjs/loader.js:653:32)","    at tryModuleLoad (internal/modules/cjs/loader.js:593:12)","    at Function.Module._load (internal/modules/cjs/loader.js:585:3)","    at Function.Module.runMain (internal/modules/cjs/loader.js:831:12)","    at startup (internal/bootstrap/node.js:283:19)"]}
START RequestId: a969d320-3033-1797-53fb-ec965c7c1866 Version: $LATEST
END RequestId: a969d320-3033-1797-53fb-ec965c7c1866
REPORT RequestId: a969d320-3033-1797-53fb-ec965c7c1866	Duration: 0.00 ms	Billed Duration: 100 ms	Memory Size: 128 MB	Max Memory Used: 42 MB	
{
  "errorType": "Runtime.ImportModuleError",
  "errorMessage": "Error: Cannot find module 'my-module'"
}

9. Test same code on windows, it's working, I can see console log printed.

10. Open Mac finder and check dependencies/nodejs/node_modules, found my-module is an Alias folder, if I delete this Alias folder and hard copy my-module folder here.

11. call sam local invoke --no-event, and check result again, it's working!

START RequestId: 565a7b74-2ea8-161a-9c1f-6c017eabce84 Version: $LATEST
2019-10-25T09:14:28.470Z	565a7b74-2ea8-161a-9c1f-6c017eabce84	INFO	this is my module
END RequestId: 565a7b74-2ea8-161a-9c1f-6c017eabce84
REPORT RequestId: 565a7b74-2ea8-161a-9c1f-6c017eabce84	Duration: 7.46 ms	Billed Duration: 100 ms	Memory Size: 128 MB	Max Memory Used: 42 MB	
{"statusCode":200,"body":"{\"message\":\"hello world\"}"}

Observed result

On Mac OS, custom module can't be required properly.

Expected result

custom module should be required successfully.

Additional environment details (Ex: Windows, Mac, Amazon Linux etc)

1. OS: Mac OS
2. sam --version:SAM CLI, version 0.23.0
3. AWS CLI version: aws-cli/1.16.266 Python/3.7.4 Darwin/18.7.0 botocore/1.13.2

Add --debug flag to command you are running

[jbeckton]

I have the same issue but with an installed module "aws-serverless-express".

When invoking a simple Lambda locally none of my imports work.

import { APIGatewayEvent, APIGatewayEventRequestContext, Context } from "aws-lambda";
const awsServerlessExpress = require( 'aws-serverless-express' );
const app = require( './app/app' );

const binaryMimeTypes = [
'application/javascript',
'application/json',
'application/octet-stream',
'application/xml',
'font/eot',
'font/opentype',
'font/otf',
'image/jpeg',
'image/png',
'image/svg+xml',
'text/comma-separated-values',
'text/css',
'text/html',
'text/javascript',
'text/plain',
'text/text',
'text/xml'
]
const server = awsServerlessExpress.createServer( app, undefined, binaryMimeTypes )
exports.handler = ( event: APIGatewayEvent, context: Context ) => 
awsServerlessExpress.proxy( server, event, context )

when I invoke the function locally I get the following error

{
  "errorType": "Runtime.ImportModuleError",
  "errorMessage": "Error: Cannot find module 'aws-serverless-express'"
}

if I change the order of the awsServerlessExpress and app imports then none of the imports in the app.js file work either.

here is the CDK code I used to define the lambda...

const goodbye = new lambda.Function( this, 'GoodbyeHandler', {
      runtime: lambda.Runtime.NODEJS_10_X,
      code: lambda.Code.fromAsset( '../lambda/goodbye/dist' ),
      handler: 'lambda.handler'
    } );

[XiamiYoung]

@jfuss can you please help look into this, I have no idea how to setup my local env in Mac now.

[jfuss]

@XiamiYoung We do not support local file references currently. You can find more details in my response here.

The current requirement is all code for the function needs to be within the CodeUri/Code property folder. When we invoke, we only mount that folder of the function. If there is code outside that folder that is needed, it is not mounted and therefore not accessible/importable/etc. You can adjust the CodeUri/Code Property to include that folder as well, copy the code into the CodeUri/Code directory, or produce a fully contained function (build) and point the CodeUri/Code to that. We have issues for supporting npm link in #756 and therefore support this in sam build. So I am going to close this in favor of that.

[XiamiYoung]

Hi @jfuss
thanks for the clarification, I have spent days on this with no progress, I'll try another approach however just out of curiosity, why this is working on windows instead?

Thank you

[Guardiannw]

If there is code outside that folder that is needed, it is not mounted and therefore not accessible/importable/etc. You can adjust the CodeUri/Code Property to include that folder as well, copy the code into the CodeUri/Code directory, or produce a fully contained function (build) and point the CodeUri/Code to that.

@jfuss are there some examples of how to make this happen while keeping the code out of the main CodeUri directory?