Alex Wein is a user on vis.social. You can follow them or interact with them if you have an account anywhere in the fediverse. If you don't, you can sign up here.
Alex Wein @alexwein
Follow

A question for the D3 pros: what tools do you use for data munging? Javascript, Spreadsheets, SQL (or an ORM), R or Python, something else? If you use JS for most of it, I'd love to know more specifics regarding libraries (or maybe you're just aces with Array.prototype methods)

@alexwein I like to work with ndjson (new line-delimited json). Bostock has some great command-line utils for transforming data in that format github.com/mbostock/ndjson-cli. For anything that’s more involved, I fall back to Python because the pydata ecosystem is so solid. 🐍

@alexwein python w/ pandas is really great for tabular data, networkx for graphs.

@alexwein I use python generally (pandas) but I've also used those ndjson tools by Mike bostock, which are JavaScript and can even be used in the terminal, but they are just a bit too cryptic for me. Pandas is quite easy to get stuff done in. I've been looking for a JavaScript data library as well though.

@alexwein I do a lot of cleaning work in Google Spreadsheets, because my datasets aren't usually enormous. When I do have something larger, I usually work through Javascript's .filter() .reduce() and .map() methods. This helped me a lot: learnjsdata.com/index.html

@alexwein almost all JS/D3 methods. slowly learning python which is great for larger sets and file export. also just came across npmjs.com/package/datalib which underpins Vega, haven't used it yet

@alexwein D3 array manipulation, Excel, OpenRefine, R, Mr Dataconverter, Mapshaper, ShapeEscape and some manual manipulation in Excel and text editor.