Webpack not working correctly with crypto module

[benedekdaniel]

We are trying to migrate to the official @google-cloud/cloud-sql-connector package in a Node.js backend (and test locally on Node 20), but we’re encountering the following error when calling connector.getOptions(...):

Support to node crypto module is required

After digging into the source, we traced this error to the node-crypto.ts module in the connector:

try {
  crypto = await import('node:crypto');
} catch (err) {
  throw new CloudSQLConnectorError({
    message: 'Support to node crypto module is required',
    code: 'ENOCRYPTOMODULE',
  });
}

We verified that require("crypto") and require("crypto").webcrypto.subtle both work fine in our runtime.

However, calling getOptions(...) fails immediately, which means the issue is triggered during dynamic import of node:crypto.

Example:

import { Connector, AuthTypes, IpAddressTypes } from "@google-cloud/cloud-sql-connector";

const connector = new Connector();

console.log("[sql-connector] crypto available:", typeof require("crypto"));
console.log("[sql-connector] connector loaded from:", require.resolve("@google-cloud/cloud-sql-connector"));

const clientOpts = await connector.getOptions({
  instanceConnectionName: "my-project:us-central1:instance",
  authType: AuthTypes.IAM,
  ipType: IpAddressTypes.PRIVATE,
});

The logs do show an available crpyto object and connector loaded from:
@google-cloud/cloud-sql-connector

Stacktrace

Support to node crypto module is required
    at Object.cryptoModule (node_modules/@google-cloud/cloud-sql-connector/build/src/node-crypto.js:26:11)

our webpack config:

const webpack = require("webpack");
const NodeExternals = require("webpack-node-externals");
const CopyWebpackPlugin = require("copy-webpack-plugin");
const TerserPlugin = require("terser-webpack-plugin");
const path = require("path");

let {
	TRAVIS_BRANCH,
	TRAVIS_COMMIT,
	TRAVIS_COMMIT_MESSAGE,
	TRAVIS_BUILD_NUMBER,
	TRAVIS_BUILD_ID,
} = process.env;

let buildInfo = {
	started: Date.now(),
	buildId: isNaN(parseInt(TRAVIS_BUILD_ID)) ? null : parseInt(TRAVIS_BUILD_ID),
	buildNumber: isNaN(parseInt(TRAVIS_BUILD_NUMBER))
		? null
		: parseInt(TRAVIS_BUILD_NUMBER),
	commitMessage: TRAVIS_COMMIT_MESSAGE || null,
	commit: TRAVIS_COMMIT || null,
	branch: TRAVIS_BRANCH || "local",
};

module.exports = {
	mode: "production",
	entry: { index: "./index.js" },
	optimization: {
		minimizer: [
			new TerserPlugin({
				terserOptions: {
					mangle: false, // <-- IMPORTANT, we use func.name for things
				},
			}),
		],
	},
	output: {
		filename: "[name].js",
		libraryTarget: "commonjs",
		devtoolModuleFilenameTemplate: "[resource-path]",
	},
	target: "node",
	module: {
		rules: [
			{
				test: /\.(jsx?)$/,
				exclude: /node_modules/,
				use: ["swc-loader"],
			},
			{
				test: /\.(tsx?)$/,
				exclude: /node_modules/,
				use: {
					loader: "swc-loader",
					options: {
						jsc: {
							parser: {
								syntax: "typescript",
							},
						},
					},
				},
			},
		],
	},
	resolve: {
		extensions: [".js", ".ts", ".json"],
		alias: {
			"@Api": path.resolve(process.cwd(), "src/api"),
			"@Models": path.resolve(process.cwd(), "src/api/models"),
			"@Types": path.resolve(process.cwd(), "src/api/types"),
			"@Consts": path.resolve(process.cwd(), "src/api/consts"),
			"@Utils": path.resolve(process.cwd(), "src/api/utils"),
			"@Functions": path.resolve(process.cwd(), "src/functions"),
		},
	},
	devtool: "source-map",
	plugins: [
		new webpack.BannerPlugin({
			raw: true,
			banner: "require('source-map-support').install();",
		}),
		new webpack.DefinePlugin({
			"process.env.BUILD_INFO": JSON.stringify(buildInfo),
		}),
		new CopyWebpackPlugin({
			patterns: [{ from: "package.json" }],
		}),
	],
	externals: [NodeExternals()],
};

webpack api config js

const { merge } = require("webpack-merge");
const webpack = require("webpack");

module.exports = merge(require("./webpack.config.js"), {
	entry: { index: "./src/api/Server.js" },
	mode: "development",
	devtool: "eval-source-map",
	output: {
		filename: "./api.js",
		libraryTarget: "this",
	},
	plugins: [
		new webpack.ProgressPlugin(),
		new webpack.DefinePlugin({
			"process.env.NODE_ENV": JSON.stringify("development"),
		}),
	],
});

Webpack api dev config js

const ESLintPlugin = require("eslint-webpack-plugin");
const { merge } = require("webpack-merge");

module.exports = merge(require("./webpack.config.js"), {
	mode: "development",
	devtool: "inline-source-map",
	optimization: {
		minimize: false,
	},
	node: {
		// allows usage of __dirname
		__dirname: false,
	},
	plugins: [
		new ESLintPlugin({
			context: "../",
			emitError: true,
			emitWarning: true,
			failOnError: true,
			extensions: ["ts", "tsx"],
			overrideConfigFile: "../functions/.eslintrc.json",
		}),
	],
});

I've tried:

• using different auth / ip types
• Checked the format of cloudConnectionName which matches the required format of: my-project:europe-west1:db-name
• Tried setting a fallback on the crypto dependency to avoid polyfills like:

fallback: {
 crypto: false,
},

//and also in externals
externals: [NodeExternals(), {crypto: false}],

• checked that the target is "node"

How to Reproduce

• Use the latest version of @google-cloud/cloud-sql-connector (1.7.0)

• Bundle your Node.js app using Webpack (target: "node", externals include webpack-node-externals)

• Call await connector.getOptions(...) as shown above

• Observe runtime error: Support to node crypto module is required

Environment details

• OS: macOS Sequoia 15.3 (24D60)

• Node.js version: v20.9.0

• npm version: 10.8.2

• @google-cloud/cloud-sql-connector: 1.7.0

• Bundler: Webpack 5 (5.93.0) with webpack-node-externals, target: "node"

• Transpiler: SWC

• Other tooling: nodemon, source-map-support, v8-compile-cache

[hessjcg]

Hi @benedekdaniel

Thank you for reporting the issue. This seems like either a webpack configuration issue or an unexpected difference between your local test environment with Node 20 and the remote runtime environment.

I created PR #436 to make this easier to troubleshoot. This will report the underlying error caused by import('node:crypto'). Would you try patching your application with this change, running it again, and then and sharing the underlying error here?

-Jonathan

[brightmileDaniel]

Hi @benedekdaniel

Thank you for reporting the issue. This seems like either a webpack configuration issue or an unexpected difference between your local test environment with Node 20 and the remote runtime environment.

I created PR #436 to make this easier to troubleshoot. This will report the underlying error caused by import('node:crypto'). Would you try patching your application with this change, running it again, and then and sharing the underlying error here?

-Jonathan

Hey @hessjcg ,

Thanks for taking the time! Even though the change above doesn't really catch the error itself I did put a breakpoint in to see if there is anything useful we can get out and this is what I found:

I get: ERR_VM_DYNAMIC_IMPORT_CALLBACK_MISSING

message is: A dynamic import callback was not specified.

stack:

"TypeError: A dynamic import callback was not specified.\n    at new NodeError (node:internal/errors:406:5)\n    at importModuleDynamicallyCallback (node:internal/modules/esm/utils:144:9)\n    at cryptoModule (/node_modules/@google-cloud/cloud-sql-connector/src/node-crypto.ts:23:5)\n    at generateKeys (/node_modules/@google-cloud/cloud-sql-connector/src/crypto.ts:22:36)\n    at CloudSQLInstance.performRefresh (/node_modules/@google-cloud/cloud-sql-connector/src/cloud-sql-instance.ts:260:48)\n    at CloudSQLInstance.refresh (/node_modules/@google-cloud/cloud-sql-connector/src/cloud-sql-instance.ts:199:16)\n    at Function.getCloudSQLInstance (/node_modules/@google-cloud/cloud-sql-connector/src/cloud-sql-instance.ts:79:20)\n    at processTicksAndRejections (node:internal/process/task_queues:95:5)"

So this is going to be a bundling / webpack issue:

We're most likely bundling code that uses dynamic import() in a CJS context, and Node doesn't know how to deal with it at runtime.

What's interesting is that I if I exclude it from the bundler it still hits the same error:

externals: [
		function ({ context, request }, callback) {
			if (request === "@google-cloud/cloud-sql-connector") {
				return callback(null, "commonjs @google-cloud/cloud-sql-connector");
			}
			return baseNodeExternals(context, request, callback);
		},
	],

[hessjcg]

Hi @benedekdaniel,

It sounds like this is a webpack issue. Unfortunately, this is beyond our area of expertise. If you find a solution that requires a change to this repo's code, let us know.