Skip to content

refactor: overhaul erpnext_sync and add test suite#1

Open
MhmdSinanKT wants to merge 1 commit into
masterfrom
shift_fetching_and_test
Open

refactor: overhaul erpnext_sync and add test suite#1
MhmdSinanKT wants to merge 1 commit into
masterfrom
shift_fetching_and_test

Conversation

@MhmdSinanKT
Copy link
Copy Markdown

Summary

Refactors the core sync script for maintainability and correctness, adds
get_shift_type_device_mapping() to fetch shift-device configuration
directly from ERPNext at startup, and introduces a full test suite with
both unit and live integration tests.


Changes

erpnext_sync.py — Refactor

New functionality

  • get_shift_type_device_mapping(): fetches Shift Type → device mappings
    from ERPNext at startup instead of requiring the user to manually maintain
    shift_type_device_mapping in local_config.py. Config is still
    supported for backward compatibility.

Bug fixes

  • send_shift_sync_to_erpnext was double-encoding the request body
    (data=json.dumps(data)json=payload), causing silent failures on
    the ERPNext side.
  • _safe_parse_dt failed to parse timestamps stored via str(datetime),
    which produces no microseconds. This caused update_shift_last_sync_timestamp
    to silently skip syncing for any device whose pull timestamp landed on a
    whole second.
  • _find_resume_index had an off-by-one risk in the original
    index_of_last + 1 pattern; replaced with a cleaner direct index return.

Refactors

  • Extracted _auth_headers() — single place to manage API credentials format
  • Extracted _resolve_punch_direction() — punch AUTO logic out of the loop
  • Extracted _find_resume_index() — resume logic now independently testable
  • Extracted _build_allowlisted_errors() — replaces module-level procedural block
  • os.path.join replaces manual '/'join([...]) string building
  • os.makedirs(exist_ok=True) replaces the if not exists guard
  • f-strings replace all "token " + x + ":" + y concatenation
  • requests.post/put/get replaces requests.request("POST", ...) calls
  • Bare except: replaced with except Exception: throughout
  • Type hints added on all public functions

test_erpnext_sync.py + conftest.py — New test suite

Unit tests (47, always run)

Class Coverage
TestSafeParseDt Valid dates, None, garbage, missing microseconds
TestSafeGetErrorStr exc extraction, plain JSON, fallback
TestResolvePunchDirection Fixed IN/OUT, AUTO detection, unknown value
TestApplyFnToKey Value transform, in-place mutation
TestGetDumpPath IP sanitisation, file extension
TestGetLastLine Missing/empty file, small/large file, multi-line
TestFindResumeIndex No history, resume after match, import start date
TestBuildAllowlistedErrors All errors default, subset by index
TestSendToErpnext 200 success, error response, correct URL for v13/v14
TestSendShiftSyncToErpnext Success, failure, connection exception
TestGetShiftTypeDeviceMapping Parsing, empty response, non-200
TestPullProcessAndPushData Full send, resume index, allowlisted/non-allowlisted errors
TestUpdateShiftLastSyncTimestamp Sync, skip conditions, min timestamp, backward compat

Live integration tests (9, opt-in via --live)

Hit a real ERPNext server using credentials from local_config.py.
All checkins created during the run are automatically deleted on teardown.

pytest test_erpnext_sync.py -v              # unit tests only
pytest test_erpnext_sync.py -v --live       # unit + live tests
pytest test_erpnext_sync.py -v -k Live      # live tests only
Class What it tests
TestLiveErpNextConnection Server reachable, credentials valid, doctype accessible
TestLiveSendToErpnext Single IN/OUT checkin, duplicate error, unknown employee error
TestLiveBatchUpload Multi-day mock attendance batch pushed end-to-end

Set LIVE_TEST_EMPLOYEE_ID at the top of the test file to pin a specific
employee, or leave as None to auto-detect the first active employee.


How to test

# Unit tests (no server needed)
pytest test_erpnext_sync.py -v

# Full suite against a running ERPNext instance
pytest test_erpnext_sync.py -v --live

- extract _auth_headers(), _resolve_punch_direction(), _find_resume_index(),
  _get_dump_path(), _build_allowlisted_errors() as standalone functions
- replace string concatenation with f-strings throughout
- fix double-encoded JSON body in send_shift_sync_to_erpnext (data= → json=)
- fix _safe_parse_dt to tolerate timestamps without microseconds
- add get_shift_type_device_mapping() to fetch shift-device config from
  ERPNext at startup instead of requiring manual config
- use os.path.join and os.makedirs(exist_ok=True) over manual path building
- replace bare except with specific exception types
- add type hints on all public functions

test: add unit and live integration test suite

- 47 unit tests covering all pure functions and mocked API calls
- 9 live integration tests behind --live flag hitting a real ERPNext server,
  with mock attendance data generation and automatic checkin cleanup
- conftest.py registers the --live pytest option
@MhmdSinanKT MhmdSinanKT requested a review from SherinKR March 7, 2026 09:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant