Open Source Heartbeat

Add kind experiment (#4)

add wip kind experiment

Signed-off-by: vsoch vsoch@users.noreply.github.com

vsoch pushed to compspec/compat-lib

tweak proot to use pwd and kill on exit

Signed-off-by: vsoch vsoch@users.noreply.github.com

vsoch commented on issue skypilot-org/skypilot#3777.

I think @romilbhardwaj gave the presentation, and I was talking to him - maybe he has a good suggestion for a next step. I do think the larger design idea of having a Kubernetes as a base is a good one, regardless of flux!…

vsoch commented on issue hanwen/go-fuse#543.

That’s perfect! …

vsoch commented on issue singularityhub/singularity-hpc#676.

As long as they are public and open, PRs to add them are welcome and encouraged!…

vsoch commented on issue hanwen/go-fuse#543.

I’d need to be able to pass the wrapper, but it won’t let me do that: …

vsoch commented on issue hanwen/go-fuse#543.

> you have to take care that a Flush operation cannot occur in parallel to any other operation that involves the Fd, or you risk a race condition (after close, the Fd may be reused for a different file.) …

vsoch pushed to flux-framework/spack

visit: add v3.4.0, v3.4.1 (#47161)

Visit: Add new versions 3.4.0 and 3.4.1
Adios2: Restrict python, 3.11 doesn’t not work for older Adios2
VisIt: Set the VTK_VERSION for @3.4:

Older versions of VTK used the VTK_{MAJOR, MINOR}_VERSION variables for VTK detection. VisIt >= 3.4 uses the full string VTK_VERSION.

CI: Don’t build llvm-amdgpu for non-HIP stack
VisIt: v3.4.1 handles newer Adios2 correctly
Visit: Add missing links in HDF5, set correct VTK version configuration parameter
VisIt: Add py-pip requirement and patch visit with configuration changes
HDF5 symlinks move when inside of callback
VisIt ninja install fails with python module. Using make does not
VisIt 3.4 has a high minimum cmake requirement
HDF5: Early return when not mpi for mpi symlinks
HDF5: Use platform agnostic method for creating legacy compatible MPI symlinks
Fix VISIT_VTK_VERSION handling for 8.2.1a hack

vsoch pushed to flux-framework/spack

Merge pull request #260 from flux-framework/update-package/flux-core-2024-11-20

Update from update-package/flux-core-2024-11-20

vsoch created a new branch, add-kind-experiment at converged-computing/lammps-time

vsoch pushed to converged-computing/jobspec-database

docs: add documentation in readme for reading databases

Signed-off-by: vsoch vsoch@users.noreply.github.com

vsoch created a new branch, wip-add-events at compspec/compat-lib

experiment: add kind cluster running results

Signed-off-by: vsoch vsoch@users.noreply.github.com

vsoch pushed to compspec/compat-lib

Merge pull request #7 from compspec/add-close

feat: add close and perfetto support

vsoch open issue compspec/compat-lib#5.

Generate perfetto output

We would want to visualize library access over time with perfetto….View Comment

vsoch pushed to vsoch/vsoch.github.io

Update 2024-11-17-across-boundaries.md

Merge pull request #280 from singularityhub/update/containers-2024-11-18

[bot] update/containers-2024-11-18

vsoch pushed to compspec/compat-lib

Merge pull request #3 from compspec/add-python-module

feat: add supporting python module

vsoch pushed to rseng/software

Merge pull request #399 from rseng/update/software-2024-11-17

Update from update/software-2024-11-17

vsoch pushed to compspec/compat-lib

add simple release workflow

This will release an x86 binary, which should be suitable for basic testing.

Signed-off-by: vsoch vsoch@users.noreply.github.com

vsoch created a new repository, rse-ops/mlperf-hpc at rse-ops/mlperf-hpc

vsoch pushed to flux-framework/spack

py-wandb: add v0.16.6 (#43891)

py-wandb: add version v0.16.6
fix: typo
py-wandb: py-click when @0.15.5:, py-pathtools when @:0.15

Co-authored-by: Wouter Deconinck wdconinc@gmail.com

vsoch open issue converged-computing/lammps-time#3.

Weekend (early) experiment plans

I’m interested to see how this looks across a more reasonably sized run, which doesn’t need to be “HPC level” but I want to last a few minutes. I want to see the degree to which patterns vary across a set of nodes (all running LAMMPS, for example) and I’m also interested in the following ideas / use cases: …View Comment

remove tag 1.1–py34 from biocontainers/reago

Signed-off-by: Vanessasaurus <814322+vsoch@users.noreply.github.com>

vsoch commented on issue flux-framework/flux-core#6437.

Oh no! But sbatch works? I

vsoch open issue converged-computing/performance-study#73.

OSU Issue

- [ ] Azure AKS missing 128 32 and 256 in plots - we have the data so must be the global function to filter is filtering it out. …View Comment

analysis: typo

Signed-off-by: vsoch vsoch@users.noreply.github.com

vsoch pushed to conda-forge/oras-py-feedstock

oras-py v0.2.25 (#30)

updated v0.2.25
MNT: Re-rendered with conda-build 24.9.0, conda-smithy 3.44.3, and conda-forge-pinning 2024.11.14.06.00.25

vsoch merged a pull request to singularityhub/shpc-registry

vsoch pushed to rseng/devstories-episodes-2

episode 102: dan reed “hpc dan”

Signed-off-by: vsoch vsoch@users.noreply.github.com

vsoch pushed to rseng/devstories

Add thank you to our HPC Dan!

vsoch pushed to rseng/devstories

episode 102: dan reed “hpc dan”

Signed-off-by: vsoch vsoch@users.noreply.github.com

vsoch commented on issue flux-framework/flux-docs#288.

I might not have the best advice (and our flux devs will step in during work hours) but I would check flux resource list to see if you actually have two nodes. It could be that running flux start a la carte like that is just getting one node. So I might try an srun in the allocation for flux start that ensures the command goes across the nodes, e.g., here is what I’m doing in a development environment with slurm (and flux installed): …

vsoch commented on issue flux-framework/flux-core#6437.

@wadudmiah I don’t see that you tried an srun command targeted at two nodes after you launched the allocation - I just see you trying an ssh command, and flux start in absence of a launching or bootstrap mechanism. Please do salloc with two nodes, and then an srun that also targets two nodes and runs flux start. The example that I provided (simplified): …

vsoch open issue converged-computing/performance-study#72.

Stream Work to Do

- [ ] Each analysis needs to be redone with the data grouped by how it was run. …View Comment

model: add markov model for predicting next path

Signed-off-by: vsoch vsoch@users.noreply.github.com

Merge pull request #277 from singularityhub/update/containers-2024-11-11

[bot] update/containers-2024-11-11

vsoch created a new branch, add-shrink-debug at researchapps/flux-sched

vsoch released 0.2.25.

## What’s Changed

Replace logger.exit(…) from the internals by @xarses in https://github.com/oras-project/oras-py/pull/155
Add a dedicated Exemption for auth handlers by @xarses in https://github.com/oras-project/oras-py/pull/158
Fix typo in simple/login.py example by @ccronca in https://github.com/oras-project/oras-py/pull/149
[tributors] contributors/update-2024-09-23 by @github-actions in https://github.com/oras-project/oras-py/pull/159
core: improve anon/auth token logic by @tarilabs in https://github.com/oras-project/oras-py/pull/148
release: 0.2.21 by @vsoch in https://github.com/oras-project/oras-py/pull/160
Allow for filepaths to include by @rhatdan in https://github.com/oras-project/oras-py/pull/161
[tributors] contributors/update-2024-10-09 by @github-actions in https://github.com/oras-project/oras-py/pull/163
core: add missing prefix property to auth backend by @tarilabs in https://github.com/oras-project/oras-py/pull/165
core: align config_path type annotation by @tarilabs in https://github.com/oras-project/oras-py/pull/166
Retry on 500 by @isinyaaa in https://github.com/oras-project/oras-py/pull/168

New Contributors

@xarses made their first contribution in https://github.com/oras-project/oras-py/pull/155
@ccronca made their first contribution in https://github.com/oras-project/oras-py/pull/149
@rhatdan made their first contribution in https://github.com/oras-project/oras-py/pull/161

Full Changelog: https://github.com/oras-project/oras-py/compare/0.2.2…0.2.25View Comment

vsoch pushed to oras-project/oras-py

Retry on 500 (#168)

workaround: retry manifest upload on quay
decorator: get rid of inheritance
decorator: retry on 500

Signed-off-by: Isabella do Amaral idoamara@redhat.com

vsoch pushed to nicholas-sly/spack

Update var/spack/repos/builtin/packages/flux-sched/package.py

Co-authored-by: Greg Becker becker33@llnl.gov

vsoch closed a pull request to flux-framework/spack

Merge pull request #278 from singularityhub/update/containers-2024-11-12

[bot] update/containers-2024-11-12

vsoch created a new branch, main at converged-computing/flux-tutorials

vsoch pushed to converged-computing/container-chonks

container times: look into specific events (#3)

container times: look into specific events
container pulling times: put run1 in the readme

Signed-off-by: vsoch vsoch@users.noreply.github.com

vsoch pushed to vsoch/vsoch.github.io

update work

Signed-off-by: vsoch vsoch@users.noreply.github.com

vsoch reviewed a singularityhub/shpc-registry pull request

None

View Review

vsoch pushed to flux-framework/spack

lua: always generate pcfile without patch and remove +pcfile variant (#47353)

lua: add +pcfile support for @5.4: versions, without using a version-dependent patch
lua: always generate pcfile, remove +pcfile variant from all packages
lua: minor fixes
rpm: minor fix

add pattern ideas to fuse analysis

Signed-off-by: vsoch vsoch@users.noreply.github.com

vsoch pushed to converged-computing/container-chonks

Merge pull request #2 from converged-computing/add-aws-pulling-study

Add aws pulling study

vsoch pushed to converged-computing/container-chonks

google: add back re-run

I did these re-runs because the settings on the kubernetes event exporting was dropping some events, and I do not think that is appropriate data to use for a publication. I am still going to run one more final study on gKE that only tests the regular pulls using their registry.

Signed-off-by: vsoch vsoch@users.noreply.github.com

vsoch pushed to rseng/software

Merge pull request #398 from rseng/update/software-2024-11-10

Update from update/software-2024-11-10

add lammps output parsing

Signed-off-by: vsoch vsoch@users.noreply.github.com

vsoch pushed to converged-computing/supermarket-fish-problem

add current/max speeds for gpu

Signed-off-by: vsoch vsoch@users.noreply.github.com

vsoch pushed to converged-computing/performance-study

Merge pull request #71 from converged-computing/scaling-governor

analysis: look at scaling governor

Merge pull request #2 from converged-computing/add-fuse-install

add copyright, notice, license

vsoch pushed to converged-computing/flux-distribute

add topology testing

Signed-off-by: vsoch vsoch@users.noreply.github.com

vsoch merged a pull request to conda-forge/sregistry-feedstock

vsoch pushed to compspec/compat-lib

feat: basic recorder functionality (#1)

feat: basic recorder functionality

We are going to want to build a bunch of hpc apps and then record what they are doing, meaning paths touched and when! This is a start.

Signed-off-by: vsoch vsoch@users.noreply.github.com

vsoch pushed to vsoch/pypi-classifiers

try ubuntu 24.04

Signed-off-by: vsoch vsoch@users.noreply.github.com

vsoch opened a pull request to converged-computing/lammps-time

vsoch pushed to compspec/compat-lib

output updates

The output log now has unix nanoseconds, and also the program exits and cleans up after the command finishes running.

Signed-off-by: vsoch vsoch@users.noreply.github.com

vsoch commented on singularityhub/singularity-hpc

Merge pull request #276 from singularityhub/update/containers-2024-11-07

[bot] update/containers-2024-11-07

vsoch pushed to rseng/gpu-search

remove partial data

I was originally saving organized based on date, but I do not anticipate doing this again so I am removing in favor of the top level.

Signed-off-by: vsoch vsoch@users.noreply.github.com

vsoch pushed to flux-framework/spack

Merge pull request #257 from flux-framework/release/flux-sched-v0.40.0

Update from release/flux-sched-v0.40.0

vsoch merged a pull request to flux-framework/flux-framework.github.io

vsoch commented on singularityhub/singularity-hpc

vsoch commented on issue flux-framework/flux-core#6411.

Thanks!…

vsoch opened a pull request to spack/spack

vsoch commented on issue singularityhub/docker2singularity#135.

Sure thing! We can try adding the arm build, but they tend to be very slow on GHA, and using singularity directly is the preferred solution I think….

vsoch commented on issue hpc-social/jobs#23.

Not that I’m aware of - as far as I understand it, their API is concerned with logging in (as a user) and then posting or sharing. https://developer.linkedin.com/product-catalog. It’s also not a good candidate for scraping because of the design of the pages, and needing to be a logged in member of the group to see said jobs. …

vsoch pushed to flux-framework/spack

Merge pull request #249 from flux-framework/release/flux-security-v0.12.0

Update from release/flux-security-v0.12.0

vsoch closed issue flux-framework/flux-operator#236.

Add topo field

We should be able to specify a custom topology, e.g.,: …View Comment

vsoch pushed to flux-framework/flux-framework.github.io

Merge pull request #130 from flux-framework/release-docs-2024-11-05

Update from release-docs-2024-11-05

vsoch pushed to converged-computing/supermarket-fish-problem

plots: restore line width

we cannot see the distribution of values without the linewidth being non-zero. I cannot remove it

Signed-off-by: vsoch vsoch@users.noreply.github.com

vsoch commented on issue converged-computing/performance-study#68.

Done….

vsoch commented on issue converged-computing/flux-views#10.

Note: this will eventually be OK to merge, but spack isn’t building a newer version of flux, it’s pinned at 0.61.2 even in noble. Likely we need to wait for the spack bases to update….

Merge pull request #275 from singularityhub/update/containers-2024-11-04

[bot] update/containers-2024-11-04

vsoch commented on issue etetoolkit/ete#762.

Awesome, thank you!…

vsoch pushed to converged-computing/performance-study

analysis: stream has incorrect title (Minife)

Signed-off-by: vsoch vsoch@users.noreply.github.com

vsoch pushed to rseng/software

Merge pull request #397 from rseng/update/software-2024-11-03

Update from update/software-2024-11-03

vsoch created a new branch, add-topology-diagram at researchapps/flux-core

vsoch commented on issue flux-framework/flux-core#6411.

> Ooh, sorry that wasn’t clear. Thanks for making it better for next intrepid explorer! …

vsoch open issue etetoolkit/ete#762.

status of library?

Hi there! :wave: This looks like a beautiful library, and one I’d like to use (we have a tree based scheduler for HPC that would be well afforded by many of these visualizations) but I cloned the repository and was trying some examples out, and there are still print statements from Python 2. I had wanted to generate the bar chart alongside the tree example and was trying to start with cluster examples, but they look like they haven’t been updated in many years. …View Comment

vsoch created a new branch, main at converged-computing/usernetes-python

vsoch pushed to converged-computing/slurm-operator

container-bases: update to rockylinux9 (#7)

container-bases: update to rockylinux9
powertools -> enable crb

Signed-off-by: vsoch vsoch@users.noreply.github.com

vsoch open issue converged-computing/fluxgen#3.

Create mode to just install flux

We might not need the config (e.g., if we are running under slurm) so it should not be required….View Comment

vsoch created a new repository, converged-computing/usernetes-python at converged-computing/usernetes-python

vsoch pushed to converged-computing/flux-usernetes

Add google to readme

vsoch open issue compose-spec/compose-go#707.

Example to read config file from string?

This is likely a more rare use case, but I want to use go embed to include a string-ified compose file (that loads into a Project) and I’m wondering if you have an example to do that? I’m looking here: https://github.com/compose-spec/compose-go/blob/b9d3b1deec9bc25de0fa81867ca4df75aef5c791/cli/options.go and I see that ReadConfigFiles can accept a path or stdin, but not a loaded string. I can try to dig deeper and see if I can come up with something, or fall back to reading from file for now, but I’m hoping someone has come up with something. Thanks!…View Comment

vsoch pushed to researchapps/flux-sched

debug: adding verbosity to grow function

We need to figure out why the function is returning -1. I am adding additional error parsing to check.

Signed-off-by: vsoch vsoch@users.noreply.github.com

vsoch created a new branch, add-shrink at researchapps/flux-sched

vsoch opened a pull request to milroy/flux-sched

vsoch opened a pull request to milroy/flux-sched

vsoch pushed to flux-framework/fluxion-go

feat: shrink support for fluxion

This changeset exposes the remove_subgraph function, which we can call a shrink. It does not account for (I do not think) handling jobs properly, but should be a reasonable start to testing or debugging.

Signed-off-by: vsoch vsoch@users.noreply.github.com

vsoch pushed to converged-computing/flux-jobset

app: add stream example

Signed-off-by: vsoch vsoch@users.noreply.github.com

vsoch pushed to converged-computing/flux-jobset

app: add kripke example on one node

Signed-off-by: vsoch vsoch@users.noreply.github.com

vsoch commented on singularityhub/singularity-hpc