Pages Router: Use of require causes build to scan parent folders.
Unanswered
goodsie posted this in #help-forum
goodsieOP
I created an endpoint to remotely execute scripts in a folder.
However when time to build the linting process is scanning parent folders of this endpoint, finding files it shouldnt and failing.
Why is the build process trying to compile anything in this destination ?
Code:
and Error:
I can't think of any reason the build process would look in any of these other folders.
However when time to build the linting process is scanning parent folders of this endpoint, finding files it shouldnt and failing.
Why is the build process trying to compile anything in this destination ?
Code:
import type { NextApiRequest, NextApiResponse } from 'next';
import { exec } from 'child_process';
export default function Endpoints(req: NextApiRequest, res: NextApiResponse) {
const ep = req.query;
const uuid = ep.endpoint;
const auid = req.headers.auth
const file = req.body.file;
const endpoint = require(`../../../processes/data/${auid}/${uuid}/${file}`);
exec(`node ${endpoint}`, (error, stdout, stderr) => {
if (error) {
res.status(500).json({ error: error.message });
return;
}
if (stderr) {
res.status(500).json({ error: stderr });
return;
}
res.status(200).json({ result: stdout });
});
}and Error:
> next build
✓ Linting and checking validity of types
Failed to compile.
./processes/data/05b3fc7d-b9d6-40d0-955b-befcb285d9ef/b09ccf66-4925-4050-b5c3-e6e246f502e7/functions/oauth/oauth2.js
Module not found: Can't resolve '../../../../../../src/Api'
https://nextjs.org/docs/messages/module-not-found
Import trace for requested module:
./processes/data/ sync ^\.\/.*\/.*\/.*$
./pages/api/ep/[endpoint].ts
./processes/data/05b3fc7d-b9d6-40d0-955b-befcb285d9ef/d36c30e5-e1a8-4eec-b577-e6308d7aa5cd/_system/utils.js
Module not found: Can't resolve 'tweetnacl'
https://nextjs.org/docs/messages/module-not-found
Import trace for requested module:
./processes/data/ sync ^\.\/.*\/.*\/.*$
./pages/api/ep/[endpoint].ts
> Build failed because of webpack errors
Creating an optimized production build .I can't think of any reason the build process would look in any of these other folders.
51 Replies
@Ray try this
ts
import fs from 'fs/promises'
const endpoint = fs.readFile(path.join(process.cwd(), 'processes', 'data', auid, uuid, file), 'utf8');
goodsieOP
Thank you Ray, This indeed fixes compile. Same as if I were to set a value.
Why would the use of requiring a file in the endpoint code cause parent folder scanning ?
Why would the use of requiring a file in the endpoint code cause parent folder scanning ?
hmm i'm not sure
goodsieOP
This has me wondering:
Import trace for requested module:
./processes/data/ sync ^\.\/.*\/.*\/.*$
./pages/api/ep/[endpoint].tshttps://vercel.com/guides/how-can-i-use-files-in-serverless-functions#next.js
this is the recommended way to read file on vercel/nextjs
this is the recommended way to read file on vercel/nextjs
goodsieOP
Yes thank you.
However, The idea here is to process a script and return a result from said script.
Previously I used a custom handler but with the switch from php to React/NextJs it seemed redundant to have 2.
In my previous solution I would require the file with no issue.
Using NextJs Pages Router Dynamic API Routing the require cause the build to scan parent folders of that required file. <-- I do not understand this.(kinda a pain in the backside too)
However, The idea here is to process a script and return a result from said script.
Previously I used a custom handler but with the switch from php to React/NextJs it seemed redundant to have 2.
In my previous solution I would require the file with no issue.
Using NextJs Pages Router Dynamic API Routing the require cause the build to scan parent folders of that required file. <-- I do not understand this.(kinda a pain in the backside too)
goodsieOP
I'd like to add that this behaviour ONLY happens when building a path as I did in the above block:
require(`../../../processes/data/${auid}/${uuid}/${file}`);why you need to use require?
goodsieOP
using require gives me a return as I would expect:
const
endpoint = require(filePath),
result = await endpoint();
res.status(200).json(result);The image above is the ideal response, if I type in the path manually I can achieve results.
what is inside the file
goodsieOP
I have been doing as much reading as possible on this, and the reason(that I can munch together) is that the webpack, due to the path being dynamic cannot compile a singular item at compile time(makes sense). so the webpack is checking all possibles at compile time
@Ray what is inside the file
goodsieOP
module.exports = async () => {
let body_ = `Hello Guest :)`;
return {
statusCode: 200,
headers: {
'Content-Type': 'text/html',
},
body: body_,
};
};its basic, test usage.
eval returns a 200 as it does not fail, however it does NOT return content (that Ive noticed)
the eval implement i used:
const result = await eval(await readFile(filePath, 'utf-8'));
res.status(200).json(result);can you use await import()
goodsieOP
hmm
did not try
@goodsie hmm
const endpoint = await import(`../../../processes/data/${auid}/${uuid}/${file}`);
exec(`node ${endpoint.default}`, (error, stdout, stderr) => {look like this work
@Ray ts
const endpoint = await import(`../../../processes/data/${auid}/${uuid}/${file}`);
exec(`node ${endpoint.default}`, (error, stdout, stderr) => {
look like this work
goodsieOP
I almost had this combination earlier.. Will give it a try.
fyi: using import like require gave the same 'module not found' error(on a perfectly fine path I might add..(...require..)
)
fyi: using import like require gave the same 'module not found' error(on a perfectly fine path I might add..(...require..)
)@Ray ts
const endpoint = await import(`../../../processes/data/${auid}/${uuid}/${file}`);
exec(`node ${endpoint.default}`, (error, stdout, stderr) => {
look like this work
goodsieOP
No jive;
When performing build it is looking in all folders that can be found inside
This is the webpack compile doing this as mentioned up above.
When performing build it is looking in all folders that can be found inside
../../../processes/data/.This is the webpack compile doing this as mentioned up above.
I'm going to play with the use of exec some more.
I do not think it is of use for me on my end however
@goodsie No jive;
When performing build it is looking in all folders that can be found inside `../../../processes/data/`.
This is the webpack compile doing this as mentioned up above.
where are you gonna deploy? I think "../../.." won't work on vercel
@Ray where are you gonna deploy? I think "../../.." won't work on vercel
goodsieOP
I was thinking the ../ would be an issue aswell.
I am deployed over aws using my own implementation of https
I am deployed over aws using my own implementation of https
I think its better to use
process.cwd() in serverless envgoodsieOP
I will try in 5 mins.
btw, Thank you for your time Ray.
goodsieOP
I went back to the filePath with path.join:
import type { NextApiRequest, NextApiResponse } from 'next';
import { access, readFile } from 'fs/promises';
import { exec } from 'child_process';
import path from 'path';
export default async function Endpoints(req: NextApiRequest, res: NextApiResponse) {
const ep = req.query;
const uuid = ep.endpoint as string;
const auid = req.headers.auth as string;
const file = req.body.file as string;
const filePath = path.join(process.cwd(), 'processes', 'data', auid, uuid, file);
const exists = await access(filePath).then(() => true).catch(() => false);
if (!exists) {
res.status(404).json({ error: 'File not found.' });
return;
}
if (req.method === 'GET') {
getEndpoint(req, res, filePath);
} else if (req.method === 'POST') {
postEndpoint(res, filePath);
} else {
res.status(405).json({ error: 'Method not allowed.' });
}
}
async function postEndpoint(res: NextApiResponse, filePath: string) {
try {
const endpoint = await import(filePath);
exec(`node ${endpoint}`, (error, stdout, stderr) => {
if (error) {
res.status(500).json({ error: error.message });
return;
}
if (stderr) {
res.status(500).json({ error: stderr });
return;
}
res.status(200).json(stdout);
});
} catch (error) {
res.status(500).json({ success: false, error: error.message });
}
};{
"success": false,
"error": "Cannot find module 'C:\\Os\\processes\\data\\05b3fc7d-b9d6-40d0-955b-befcb285d9ef\\b09ccf66-4925-4050-b5c3-e6e246f502e7\\endpoints\\hello.js'",
"filePath": "C:\\Os\\processes\\data\\05b3fc7d-b9d6-40d0-955b-befcb285d9ef\\b09ccf66-4925-4050-b5c3-e6e246f502e7\\endpoints\\hello.js"
}Paths are correct, can be pasted to file browser to achieve file( I also doubt this every 20 seconds 😅 )
@goodsie I went back to the filePath with path.join:
js
import type { NextApiRequest, NextApiResponse } from 'next';
import { access, readFile } from 'fs/promises';
import { exec } from 'child_process';
import path from 'path';
export default async function Endpoints(req: NextApiRequest, res: NextApiResponse) {
const ep = req.query;
const uuid = ep.endpoint as string;
const auid = req.headers.auth as string;
const file = req.body.file as string;
const filePath = path.join(process.cwd(), 'processes', 'data', auid, uuid, file);
const exists = await access(filePath).then(() => true).catch(() => false);
if (!exists) {
res.status(404).json({ error: 'File not found.' });
return;
}
if (req.method === 'GET') {
getEndpoint(req, res, filePath);
} else if (req.method === 'POST') {
postEndpoint(res, filePath);
} else {
res.status(405).json({ error: 'Method not allowed.' });
}
}
async function postEndpoint(res: NextApiResponse, filePath: string) {
try {
const endpoint = await import(filePath);
exec(`node ${endpoint}`, (error, stdout, stderr) => {
if (error) {
res.status(500).json({ error: error.message });
return;
}
if (stderr) {
res.status(500).json({ error: stderr });
return;
}
res.status(200).json(stdout);
});
} catch (error) {
res.status(500).json({ success: false, error: error.message });
}
};
import { exec } from "child_process";
import { access } from "fs/promises";
import { NextApiRequest, NextApiResponse } from "next";
import path from "path";
import { promisify } from "util";
import JSON5 from "json5";
const execPromise = promisify(exec);
export default async function handler(
req: NextApiRequest,
res: NextApiResponse
) {
try {
const filePath = path.join(process.cwd(), "data.js");
const exists = await access(filePath)
.then(() => true)
.catch(() => false);
if (!exists) {
res.status(404).json({ error: "File not found." });
return;
}
const out = await execPromise(`node ${filePath}`);
const data = JSON5.parse(out.stdout);
res.status(200).json({ success: true, data });
} catch (error) {
res.status(500).json({ success: false, error: (error as Error).message });
}
}// data.js
(() => {
let body_ = `Hello Guest :)`;
console.log({
statusCode: 200,
headers: {
"Content-Type": "text/html",
},
body: body_,
});
})();got it work
@Ray js
// data.js
(() => {
let body_ = `Hello Guest :)`;
console.log({
statusCode: 200,
headers: {
"Content-Type": "text/html",
},
body: body_,
});
})();
goodsieOP
I think this was biggest of changes. i was getting empty strings before. both exec methods. Changing file on endpoint gave the return
I think there is not necessary to import
Thank You, Very Much
goodsieOP
used to be module.exports = async () =>{}
so yes
yes this would not work lol
goodsieOP
it does with require 😉
lol
ow
goodsieOP
I now have a method and pre-determined environment for user endpoint handlers.
+100
+100