New top story on Hacker News: Show HN: Pyproc – Call Python from Go Without CGO or Microservices

Show HN: Pyproc – Call Python from Go Without CGO or Microservices
3 by acc_10000 | 0 comments on Hacker News.
Hi HN!I built *pyproc* to let Go services call Python like a local function — *no CGO and no separate microservice*. It runs a pool of Python worker processes and talks over *Unix Domain Sockets* on the same host/pod, so you get low overhead, process isolation, and parallelism beyond the GIL. *Why this exists* * Keep your Go service, reuse Python/NumPy/pandas/PyTorch/scikit-learn. * Avoid network hops, service discovery, and ops burden of a separate Python service. *Quick try (\~5 minutes)* Go (app): ``` go get github.com/YuminosukeSato/pyproc@latest ``` Python (worker): ``` pip install pyproc-worker ``` Minimal worker (Python): ``` from pyproc_worker import expose, run_worker @expose def predict(req): return {"result": req["value"] * 2} if __name__ == "__main__": run_worker() ``` Call from Go: ``` import ( "context" "fmt" "github.com/YuminosukeSato/pyproc/pkg/pyproc" ) func main() { pool, _ := pyproc.NewPool(pyproc.PoolOptions{ Config: pyproc.PoolConfig{Workers: 4, MaxInFlight: 10}, WorkerConfig: pyproc.WorkerConfig{SocketPath: "/tmp/pyproc.sock", PythonExec: "python3", WorkerScript: "worker.py"}, }, nil) _ = pool.Start(context.Background()) defer pool.Shutdown(context.Background()) var out map[string]any _ = pool.Call(context.Background(), "predict", map[string]any{"value": 42}, &out) fmt.Println(out["result"]) // 84 } ``` *Scope / limits* * Same-host/pod only (UDS). Linux/macOS supported; Windows named pipes not yet. * Best for request/response payloads ≲ \~100 KB JSON; GPU orchestration and cross-host serving are out of scope. *Benchmarks (indicative)* * Local M1, simple JSON: \~*45µs p50* and *\~200k req/s* with 8 workers. Your numbers will vary. *What’s included* * Pure Go client (no CGO), Python worker lib, pool, health checks, graceful restarts, and examples. *Docs & code* * README, design/ops/security docs, pkg.go.dev: [ https://ift.tt/sp5cgAL ]( https://ift.tt/sp5cgAL ) *License* * Apache-2.0. Current release: v0.2.x. *Feedback welcome* * API ergonomics, failure modes under load, and priorities for codecs/transports (e.g., Arrow IPC, gRPC-over-UDS). --- Source for details: project README and docs. ([github.com][1]) [1]: https://ift.tt/sp5cgAL "GitHub - YuminosukeSato/pyproc: Call Python from Go without CGO or microservices - Unix domain socket based IPC for ML inference and data processin"