Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Release Worker 1.13.0 #907

Merged
merged 3 commits into from
Mar 19, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions integration-tests/execute/CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,12 @@
# @openfn/integration-tests-execute

## 1.0.18

### Patch Changes

- @openfn/[email protected]
- @openfn/[email protected]

## 1.0.17

### Patch Changes
Expand Down
2 changes: 1 addition & 1 deletion integration-tests/execute/package.json
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"name": "@openfn/integration-tests-execute",
"private": true,
"version": "1.0.17",
"version": "1.0.18",
"description": "Job execution tests",
"author": "Open Function Group <[email protected]>",
"license": "ISC",
Expand Down
12 changes: 12 additions & 0 deletions integration-tests/worker/CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,17 @@
# @openfn/integration-tests-worker

## 1.0.81

### Patch Changes

- Updated dependencies [0a176aa]
- Updated dependencies [0a176aa]
- Updated dependencies [ce5022a]
- @openfn/[email protected]
- @openfn/[email protected]
- @openfn/[email protected]
- @openfn/[email protected]

## 1.0.80

### Patch Changes
Expand Down
2 changes: 1 addition & 1 deletion integration-tests/worker/package.json
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"name": "@openfn/integration-tests-worker",
"private": true,
"version": "1.0.80",
"version": "1.0.81",
"description": "Lightning WOrker integration tests",
"author": "Open Function Group <[email protected]>",
"license": "ISC",
Expand Down
2 changes: 1 addition & 1 deletion integration-tests/worker/test/exit-reasons.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -121,7 +121,7 @@ test('exception: bad credential (not found)', async (t) => {
t.is(error_type, 'CredentialLoadError');
t.is(
error_message,
'Failed to load credential been-to-the-mountain: not_found'
'Failed to load credential been-to-the-mountain: [fetch:credential] not_found'
);
});

Expand Down
27 changes: 26 additions & 1 deletion integration-tests/worker/test/integration.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -313,7 +313,7 @@ test.serial('run a job with bad credentials', (t) => {
t.is(payload.error_type, 'CredentialLoadError');
t.regex(
payload.error_message,
/Failed to load credential zzz: not_found/
/Failed to load credential zzz: \[fetch:credential\] not_found/
);
done();
});
Expand Down Expand Up @@ -561,6 +561,31 @@ test.serial("Don't send job logs to stdout", (t) => {
});
});

// This is a test against job logs - but it should work
// exactly the same way for all logs
test.serial("Don't send empty logs to lightning", (t) => {
return new Promise(async (done) => {
const attempt = {
id: crypto.randomUUID(),
jobs: [
{
adaptor: '@openfn/language-common@latest',
body: 'fn((s) => { console.log(); return s })',
},
],
};

lightning.once('run:complete', () => {
// The engine logger shouldn't print out any job logs
const jobLogs = engineLogger._history.filter((l) => l.name === 'JOB');
t.is(jobLogs.length, 0);
done();
});

lightning.enqueueRun(attempt);
});
});

test.serial('Include timestamps on basically everything', (t) => {
return new Promise(async (done) => {
const attempt = {
Expand Down
11 changes: 11 additions & 0 deletions packages/cli/CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,16 @@
# @openfn/cli

## 1.11.4

### Patch Changes

- Updated dependencies [0a176aa]
- @openfn/[email protected]
- @openfn/[email protected]
- @openfn/[email protected]
- @openfn/[email protected]
- @openfn/[email protected]

## 1.11.3

### Patch Changes
Expand Down
2 changes: 1 addition & 1 deletion packages/cli/package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "@openfn/cli",
"version": "1.11.3",
"version": "1.11.4",
"description": "CLI devtools for the openfn toolchain.",
"engines": {
"node": ">=18",
Expand Down
8 changes: 8 additions & 0 deletions packages/compiler/CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,13 @@
# @openfn/compiler

## 1.0.2

### Patch Changes

- Updated dependencies [0a176aa]
- @openfn/[email protected]
- @openfn/[email protected]

## 1.0.1

### Patch Changes
Expand Down
2 changes: 1 addition & 1 deletion packages/compiler/package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "@openfn/compiler",
"version": "1.0.1",
"version": "1.0.2",
"description": "Compiler and language tooling for openfn jobs.",
"author": "Open Function Group <[email protected]>",
"license": "ISC",
Expand Down
7 changes: 7 additions & 0 deletions packages/deploy/CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,12 @@
# @openfn/deploy

## 0.11.1

### Patch Changes

- Updated dependencies [0a176aa]
- @openfn/[email protected]

## 0.11.0

### Minor Changes
Expand Down
2 changes: 1 addition & 1 deletion packages/deploy/package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "@openfn/deploy",
"version": "0.11.0",
"version": "0.11.1",
"description": "Deploy projects to Lightning instances",
"type": "module",
"exports": {
Expand Down
10 changes: 10 additions & 0 deletions packages/engine-multi/CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,15 @@
# engine-multi

## 1.6.2

### Patch Changes

- Updated dependencies [0a176aa]
- @openfn/[email protected]
- @openfn/[email protected]
- @openfn/[email protected]
- @openfn/[email protected]

## 1.6.1

### Patch Changes
Expand Down
2 changes: 1 addition & 1 deletion packages/engine-multi/package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "@openfn/engine-multi",
"version": "1.6.1",
"version": "1.6.2",
"description": "Multi-process runtime engine",
"main": "dist/index.js",
"type": "module",
Expand Down
23 changes: 12 additions & 11 deletions packages/engine-multi/test/integration.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -263,7 +263,6 @@ test.serial('run without error if no state is returned', (t) => {

api.execute(plan, emptyState).on('workflow-complete', ({ state }) => {
t.falsy(state);
t.log(logger._history);

// Ensure there are no error logs
const err = logger._find('error', /./);
Expand Down Expand Up @@ -532,17 +531,19 @@ export default [(state) => {
},
]);
const options = {
payloadLimitMb: 0.5,
payloadLimitMb: 0.1,
};

api.execute(plan, emptyState, options).on('workflow-log', (evt) => {
if (evt.name === 'JOB') {
t.deepEqual(evt.message, REDACTED_LOG.message);
}
});

api.execute(plan, emptyState, options).on('workflow-complete', () => {
done();
});
api
.execute(plan, emptyState, options)
.on('workflow-log', (evt) => {
console.log(evt);
if (evt.name === 'JOB') {
t.deepEqual(evt.message, REDACTED_LOG.message);
}
})
.on('workflow-complete', () => {
done();
});
});
});
10 changes: 10 additions & 0 deletions packages/lightning-mock/CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,15 @@
# @openfn/lightning-mock

## 2.1.4

### Patch Changes

- Updated dependencies [0a176aa]
- @openfn/[email protected]
- @openfn/[email protected]
- @openfn/[email protected]
- @openfn/[email protected]

## 2.1.3

### Patch Changes
Expand Down
2 changes: 1 addition & 1 deletion packages/lightning-mock/package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "@openfn/lightning-mock",
"version": "2.1.3",
"version": "2.1.4",
"private": true,
"description": "A mock Lightning server",
"main": "dist/index.js",
Expand Down
5 changes: 5 additions & 0 deletions packages/lightning-mock/src/api-sockets.ts
Original file line number Diff line number Diff line change
Expand Up @@ -407,6 +407,11 @@ const createSocketAPI = (
}
state.pending[runId].steps[job_id] = step_id;

// let payload: any = {
// status: 'error',
// response: 'test_error',
// };

let payload: any = {
status: 'ok',
};
Expand Down
6 changes: 6 additions & 0 deletions packages/logger/CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,11 @@
# @openfn/logger

## 1.0.5

### Patch Changes

- 0a176aa: In JSON mode, don't log empty messages

## 1.0.4

### Patch Changes
Expand Down
2 changes: 1 addition & 1 deletion packages/logger/package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "@openfn/logger",
"version": "1.0.4",
"version": "1.0.5",
"description": "Cross-package logging utility",
"module": "dist/index.js",
"author": "Open Function Group <[email protected]>",
Expand Down
8 changes: 8 additions & 0 deletions packages/logger/src/logger.ts
Original file line number Diff line number Diff line change
Expand Up @@ -158,6 +158,14 @@ export default function (name?: string, options: LogOptions = {}): Logger {
level: LogFns,
...args: LogArgs
) => {
if (args.length === 0) {
// In JSON mode, don't log empty lines
// (mostly because this spams Lightning with nonsense, but more generally
// if you have machine readable logs then "whitespace" or "formatting" logs,
// like console.break(), are meaningless)
return;
}

const message = args.map((o) =>
sanitize(o, {
stringify: false,
Expand Down
11 changes: 11 additions & 0 deletions packages/logger/test/logger.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -354,6 +354,17 @@ test('json mode should serialize errors nicely', (t) => {
t.deepEqual(result.message[0], { name: 'Error', message: 'wibble' });
});

test('json mode should not log empty lines', (t) => {
const logger = createLogger<JSONLog>(undefined, {
level: 'info',
json: true,
});

logger.info();

t.is(logger._history.length, 0);
});

test('with level=default, logs success, error and warning but not info and debug', (t) => {
const logger = createLogger<StringLog>('x', { level: 'default' });

Expand Down
7 changes: 7 additions & 0 deletions packages/runtime/CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,12 @@
# @openfn/runtime

## 1.6.4

### Patch Changes

- Updated dependencies [0a176aa]
- @openfn/[email protected]

## 1.6.3

### Patch Changes
Expand Down
2 changes: 1 addition & 1 deletion packages/runtime/package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "@openfn/runtime",
"version": "1.6.3",
"version": "1.6.4",
"description": "Job processing runtime.",
"type": "module",
"exports": {
Expand Down
15 changes: 15 additions & 0 deletions packages/ws-worker/CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,20 @@
# ws-worker

## 1.13.0

### Minor Changes

- ce5022a: Added sentry notifications for server and websocket errors

### Patch Changes

- 0a176aa: Ignore empty log lines (don't send them to lightning)
- Updated dependencies [0a176aa]
- @openfn/[email protected]
- @openfn/[email protected]
- @openfn/[email protected]
- @openfn/[email protected]

## 1.12.1

### Patch Changes
Expand Down
4 changes: 3 additions & 1 deletion packages/ws-worker/package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "@openfn/ws-worker",
"version": "1.12.1",
"version": "1.13.0",
"description": "A Websocket Worker to connect Lightning to a Runtime Engine",
"main": "dist/index.js",
"type": "module",
Expand All @@ -25,6 +25,7 @@
"@openfn/lexicon": "workspace:^",
"@openfn/logger": "workspace:*",
"@openfn/runtime": "workspace:*",
"@sentry/node": "^9.5.0",
"@types/koa-logger": "^3.1.2",
"@types/ws": "^8.5.6",
"fast-safe-stringify": "^2.1.1",
Expand All @@ -48,6 +49,7 @@
"@types/yargs": "^17.0.12",
"ava": "5.1.0",
"nodemon": "3.0.1",
"sentry-testkit": "^6.1.0",
"tslib": "^2.4.0",
"tsup": "^6.2.3",
"typescript": "^4.6.4",
Expand Down
Loading