Central idea. The three tools address different stages of empirical work.
editanythinghelps us inspect files without leaving Stata;combineallconverts or combines every relevant file in a directory; and Stata frames allow several datasets to coexist, remain separate, and communicate in memory.
1. A useful end-of-week habit
At the end of each week, I usually look at the additions and updates available through the Statistical Software Components archive. In Stata, the natural starting point is:
ssc new
This week, two packages attracted my attention: COMBINEALL and EDITANYTHING. Both packages are written by Eric A. Booth and Elizabeth Teas and were recorded on SSC on July 30, 2026.
The two commands may initially look like small utilities. In practice, they address a recurrent source of friction in empirical research: locating files, inspecting their contents, converting heterogeneous formats, combining many data releases, and keeping intermediate datasets available for validation.
The native frames architecture makes this workflow especially interesting. Frames are not a new 2026 feature: they were introduced in Stata 16. Their importance is that one Stata session can hold several datasets in memory simultaneously.
| Layer | Main question | Tool |
|---|---|---|
| Files on disk | How can I inspect or edit this source file? | editanything |
| Batch data management | How can I process every relevant file in a directory? | combineall |
| Datasets in memory | How can I work with several datasets simultaneously? | Stata frames |
2. Installing and checking the packages
The packages can be installed or updated directly from SSC:
ssc install combineall, replace
ssc install editanything, replace
which combineall
which editanything
help combineall
help editanything
The current files inspected for this post are combineall version 2.0.1 and editanything version 2.0.0.
3. Inspecting files with EDITANYTHING
editanything generalizes the logic of commands such as adoedit. Instead of being restricted to an ado-file, it can locate and open almost any plain-text file: Stata programs and help files, CSV data, logs, Markdown documents, LaTeX and BibTeX files, Python and R scripts, SQL queries, and configuration files.
The default action is to open the file in Stata’s Do-file Editor. Other modes are available:
| Option | Behavior |
|---|---|
editor | Open the file in the Do-file Editor; this is the default |
view | Open the file read-only in Stata’s Viewer |
external | Open the file with the operating system’s default application |
showpath | Resolve and display the file path without opening it |
clipboard | Copy the raw contents to the system clipboard |
new | Create a new empty file and open it |
For example, we can find the installed source of combineall without opening it:
editanything combineall, showpath
return list
display "`r(file)'"
display "`r(extension)'"
display r(size)
The command searches the current directory and the ado-path. It returns the resolved path in r(file), the extension in r(extension), the filename in r(basename), and usually the file size in r(size).
We can open the package’s help source in Stata’s Do-file Editor:
editanything combineall.sthlp, editor
We can also inspect a raw CSV file without importing it:
global CA_ROOT "C:/Users/jamel/Dropbox/stata/combineall"
global CA_INPUTS "$CA_ROOT/grunfeld_merge_inputs"
editanything "$CA_INPUTS/investment.csv", editor
With the
editormode—which is also the default—EDITANYTHING opens the CSV file as raw text in Stata’s Do-file Editor. This is not the same as importing the file: no Stata dataset or frame is created, and the data currently in memory remain unchanged. The command is useful for inspecting headers, delimiters, variable names, and the underlying file structure without leaving Stata.
A new project note can also be created directly from Stata:
cd "$CA_ROOT"
editanything workflow_notes, new extension(md)
The new option intentionally returns an error if the file already exists. This prevents an existing document from being overwritten inadvertently.
For binary files such as PDF, XLSX, PNG, or DTA files, the appropriate mode is generally external. Very large text files may also be more comfortable in an external editor.
The command includes safeguards against directly changing Stata-supplied programs. Editing files in Stata’s BASE or UPDATES directories with force is particularly risky because an update can erase the modifications. For serious programming, it is safer to study the source and create a separately named program.
4. Understanding COMBINEALL
combineall processes every file of a selected type in a directory. It recognizes native Stata datasets, CSV and other delimited text files, Excel workbooks, and XML files.
The most important choice is cmethod():
| Method | Operation | Typical application |
|---|---|---|
convertonly | Convert each input into a separate DTA file | Convert a directory of CSV or Excel files |
append | Add observations vertically | Stack annual releases containing the same variables |
merge | Add variables horizontally using identifiers | Combine different variables for the same country-years or firm-years |
joinby | Form pairwise combinations within common groups | Combine nonunique observations within matching groups |
The default method is convertonly, and the default file type is CSV. Consequently, a bare command converts every CSV file in the working directory into a separate Stata dataset:
combineall
For reproducible work, I prefer to make the choices explicit:
combineall, ///
cmethod(convertonly) ///
directory("$CA_INPUTS") ///
filetype(csv) ///
replace
Use a dedicated input directory.
combineallprocesses every file matchingfiletype()in the specified directory. It does not select files from an arbitrary inclusion list.
5. A panel-data merge with the Grunfeld data
To illustrate the distinction between appending and merging, consider Stata’s standard Grunfeld panel. It contains investment, market value, and capital-stock data for 10 companies observed over 20 years.
We will split the original panel into three CSV files:
investment.csv: firm, year, and investment;market_value.csv: firm, year, and market value;capital_stock.csv: firm, year, and capital stock.
Each file contains the same 200 firm-year observations but a different economic variable. The appropriate operation is therefore a one-to-one merge. Appending the files would incorrectly produce 600 observations with many missing cells.
Step 1: Define persistent project paths
Global macros are used deliberately so that later blocks can be rerun separately once the project paths have been defined. Blocks that use generated files still require the corresponding file-creation steps to have been executed at least once.
version 16.0
clear all
set more off
global CA_ROOT "C:/Users/jamel/Dropbox/stata/combineall"
global CA_INPUTS "$CA_ROOT/grunfeld_merge_inputs"
global CA_MERGED "$CA_ROOT/grunfeld_merged_raw.dta"
global CA_FINAL "$CA_ROOT/grunfeld_panel_final.dta"
capture mkdir "$CA_ROOT"
capture mkdir "$CA_INPUTS"
Step 2: Create the component files
webuse grunfeld, clear
describe
isid company year
assert _N == 200
/*
combineall currently requires string keys for merge and joinby.
The F and Y prefixes ensure that import delimited retains the
identifiers as strings.
*/
gen str3 firm_id = "F" + strtrim(strofreal(company))
gen str5 year_id = "Y" + strtrim(strofreal(year))
isid firm_id year_id
preserve
keep firm_id year_id invest
export delimited using "$CA_INPUTS/investment.csv", replace
restore
preserve
keep firm_id year_id mvalue
export delimited using "$CA_INPUTS/market_value.csv", replace
restore
preserve
keep firm_id year_id kstock
export delimited using "$CA_INPUTS/capital_stock.csv", replace
restore
dir "$CA_INPUTS/*.csv"
The prefixes are important. Without them, import delimited could interpret identifiers such as company and year as numeric variables. The current combineall merge engine creates string key variables internally, so numeric merge keys would generate a type mismatch.
Step 3: Merge every CSV file
combineall using "$CA_MERGED", ///
cmethod(merge) ///
directory("$CA_INPUTS") ///
filetype(csv) ///
mtype(1:1) ///
mvars(firm_id year_id) ///
replace
return list
The command discovers the three CSV files, converts them internally, and successively merges them using firm_id and year_id.
An important feature is that combineall writes the merged dataset to the file specified after using and restores the dataset that was already in memory. It does not automatically load the merged result.
Step 4: Reconstruct and validate the final panel
use "$CA_MERGED", clear
// Verify that the final merge produced complete matches
assert _merge == 3
drop _merge
gen byte company = real(subinstr(firm_id, "F", "", 1))
gen int year = real(subinstr(year_id, "Y", "", 1))
order company year firm_id year_id invest mvalue kstock
sort company year
isid company year
assert _N == 200
assert !missing(invest, mvalue, kstock)
xtset company year
xtdescribe
list company year invest mvalue kstock in 1/12, ///
sepby(company)
compress
save "$CA_FINAL", replace
The final validation is essential. Automation reduces repetitive code, but it does not prove that the merge keys are correct. Here, we verify that:
- the firm-year key uniquely identifies every observation;
- the merged dataset contains exactly 200 observations;
- none of the three economic variables is missing; and
- the result is a strongly balanced panel of 10 companies over 20 years.
6. An advanced feature for annual vintages
The most interesting 2026 addition to combineall may be its harmonization layer for repeated annual releases. Official statistical agencies frequently change variable names between vintages. A direct append can then create several columns for what is economically the same variable.
The map() option accepts a CSV crosswalk with four columns:
oldname,newname,firstyear,lastyear
gdp_old,gdp,1990,2009
gdp_usd,gdp,2010,
The package extracts a four-digit year from each filename, applies only the renames relevant to that vintage, creates a numeric year variable, and records the original variable name in a Stata characteristic.
combineall using "$CA_ROOT/macro_panel.dta", ///
cmethod(append) ///
directory("$CA_ROOT/annual_releases") ///
filetype(csv) ///
map("$CA_ROOT/renames.csv") ///
strict ///
replace
return list
// The example should process exactly three annual releases
assert r(n_files) == 3
assert r(n_missing) == 0The strict option converts an expected-but-absent mapped variable into an error. This can be valuable when the crosswalk is treated as a formal data contract rather than an informal cleaning aid.
7. Where Stata frames enter the workflow
A frame is a named container holding one Stata dataset in memory. Only one frame is current, meaning that ordinary Stata commands operate on it, but commands can be sent to other frames with the frame name: prefix.
| Command | Meaning |
|---|---|
frame create name | Create an empty named frame |
frame change name | Make that frame current |
frame name: command | Run one command in another frame and return automatically |
frame copy source target | Create an independent copy of a complete dataset |
frame put ..., into(target) | Copy selected variables or observations |
frlink | Create a directional mapping between observations in two frames |
frget | Copy selected variables through an existing link |
We can load the three Grunfeld component files into separate frames:
/*
Start this block only after saving any unsaved work:
clear all removes all existing frames.
*/
clear all
global CA_ROOT "C:/Users/jamel/Dropbox/stata/combineall"
global CA_INPUTS "$CA_ROOT/grunfeld_merge_inputs"
frame rename default investment
frame investment: import delimited using ///
"$CA_INPUTS/investment.csv", clear
frame create market
frame market: import delimited using ///
"$CA_INPUTS/market_value.csv", clear
frame create capital
frame capital: import delimited using ///
"$CA_INPUTS/capital_stock.csv", clear
frames dir
Loading the market-value and capital-stock files does not replace the investment data. The three datasets coexist in memory, and each can be inspected or transformed independently.
Linking the component datasets
Because every firm-year appears once in each file, the relationship between the frames is one-to-one:
frame investment: isid firm_id year_id
frame market: isid firm_id year_id
frame capital: isid firm_id year_id
frame change investment
frlink 1:1 firm_id year_id, frame(market)
frget mvalue, from(market)
frlink 1:1 firm_id year_id, frame(capital)
frget kstock, from(capital)
assert !missing(invest, mvalue, kstock)
gen byte company = real(subinstr(firm_id, "F", "", 1))
gen int year = real(subinstr(year_id, "Y", "", 1))
xtset company year
describe
summarize invest mvalue kstock
frlink does not merge the datasets. It constructs an observation map between the current frame and another frame. The frget command then copies only the requested variables through that map.
Each frlink command also creates a link variable in the current frame. In this example, these variables are named market and capital. They should be retained when saving the complete frameset because they contain the observation mappings. When producing an ordinary standalone DTA file, however, we can copy only the substantive panel variables into a separate frame.
This distinction is useful. combineall is convenient when the objective is to process an entire directory and produce a durable combined dataset on disk. Frames are preferable when the source datasets should remain separate in memory and only selected variables are needed.
In Stata 18 or later, fralias add offers a memory-saving alternative to frget. Instead of physically copying a variable, it creates a read-only reference to the variable stored in the linked frame.
Stata 18 also introduced framesets, which allow a collection of related frames to be saved together in a .dtas file:
frames save "$CA_ROOT/grunfeld_workflow", ///
frames(investment market capital) ///
replace
8. Practical safeguards
- Separate the input files. Use a dedicated directory because
combineallprocesses every file matchingfiletype(). - Choose the correct operation. Append adds observations; merge adds variables. They are not interchangeable.
- Check the key variables. Use
isidbefore and after a one-to-one merge. - Remember the current string-key limitation. The present merge and
joinbyengine requires string key variables. - Validate observation counts and missing values. A command can finish successfully even when the economic structure is wrong.
- Be careful with
append. The current engine usesappend, force, so numeric/string conflicts can become missing values rather than stopping execution. - Avoid unnecessary
tostring. Its use of display formats can discard numeric precision. It is more appropriate for identifier-like variables than measured quantities. - Protect native DTA files. With
filetype(dta), specify a prefix or suffix when transformations could otherwise rewrite source files in place. - Do not use EDITANYTHING as a data-cleaning shortcut. Manual changes to a CSV are difficult to audit. Substantive transformations belong in a do-file.
- Save existing work before resetting frames. Commands such as
clear allandframes resetremove every frame in memory.
Conclusion
These tools are useful because they reduce the friction surrounding empirical work. editanything makes the project’s files easier to inspect. combineall automates repeated conversion, append, merge, and harmonization tasks. Frames allow component datasets and final results to coexist and be compared without repeatedly loading and replacing data.
The resulting workflow is simple: inspect the source, automate the combination, and validate the result across frames. That is where the fun begins.
References and documentation
- Booth, Eric A., and Elizabeth Teas. 2026. COMBINEALL: Stata module to combine, append, merge, joinby, or convert every file in a directory.
- Booth, Eric A., and Elizabeth Teas. 2026. EDITANYTHING: Stata module to open any text file from Stata.
- StataCorp. Data frames: multiple datasets in memory.
- StataCorp. Introduction to frames.
- StataCorp. Frame sets.
Full code
version 16.0
cls
clear all
set more off
/*
Execution dependencies
- Run Part 0 first to install the packages and define the globals.
- Parts 2 and 3 require the component CSV files created in Part 1.
- Part 4 requires the merged file produced in Part 3.
- Part 7 requires the annual CSV files and rename map created in
Parts 5 and 6.
- Part 8 requires the component CSV files created in Part 1.
- Part 9 requires the frames created in Part 8.
- Part 10 requires the linked frames produced in Parts 8 and 9.
*/
// =============================================================================
// PART 0 — INSTALL THE PACKAGES AND DEFINE THE PROJECT PATHS
// =============================================================================
// Install or update the packages
ssc install combineall, replace
ssc install editanything, replace
which combineall
which editanything
// Main project directories and files
global CA_ROOT "C:/Users/jamel/Dropbox/stata/combineall"
global CA_INPUTS "$CA_ROOT/grunfeld_merge_inputs"
global CA_MERGED "$CA_ROOT/grunfeld_merged_raw.dta"
global CA_FINAL "$CA_ROOT/grunfeld_panel_final.dta"
// Files used for the harmonization example
global CA_ANNUAL "$CA_ROOT/grunfeld_annual_inputs"
global CA_MAP "$CA_ROOT/grunfeld_rename_map.csv"
global CA_HARMONIZED "$CA_ROOT/grunfeld_harmonized_1935_1937.dta"
// Files produced by the frames example
global CA_FRAME_FINAL "$CA_ROOT/grunfeld_panel_from_frames.dta"
global CA_FRAMESET "$CA_ROOT/grunfeld_workflow"
// Create the required directories
capture mkdir "$CA_ROOT"
capture mkdir "$CA_INPUTS"
capture mkdir "$CA_ANNUAL"
// Display the project paths
display "$CA_ROOT"
display "$CA_INPUTS"
// =============================================================================
// PART 1 — CREATE THREE COMPONENT FILES FROM THE GRUNFELD PANEL
// =============================================================================
webuse grunfeld, clear
describe
isid company year
assert _N == 200
/*
combineall currently requires string identifiers for merge and joinby.
The F and Y prefixes ensure that import delimited retains the identifiers
as strings rather than interpreting them as numeric variables.
*/
gen str3 firm_id = "F" + strtrim(strofreal(company))
gen str5 year_id = "Y" + strtrim(strofreal(year))
order company year firm_id year_id invest mvalue kstock
isid firm_id year_id
// -----------------------------------------------------------------------------
// Investment file
// -----------------------------------------------------------------------------
preserve
keep firm_id year_id invest
export delimited using ///
"$CA_INPUTS/investment.csv", replace
restore
// -----------------------------------------------------------------------------
// Market-value file
// -----------------------------------------------------------------------------
preserve
keep firm_id year_id mvalue
export delimited using ///
"$CA_INPUTS/market_value.csv", replace
restore
// -----------------------------------------------------------------------------
// Capital-stock file
// -----------------------------------------------------------------------------
preserve
keep firm_id year_id kstock
export delimited using ///
"$CA_INPUTS/capital_stock.csv", replace
restore
// Verify the files
dir "$CA_INPUTS/*.csv"
// =============================================================================
// PART 2 — INSPECT THE FILES WITH EDITANYTHING
// =============================================================================
// Find the installed combineall ado-file without opening it
editanything combineall, showpath
return list
display "`r(file)'"
display "`r(basename)'"
display "`r(extension)'"
display r(size)
// Open the combineall help source in the Do-file Editor
editanything combineall.sthlp, editor
// Display the raw CSV file without importing it
editanything "$CA_INPUTS/investment.csv", editor
// Display the raw CSV file without importing it
editanything "$CA_INPUTS/capital_stock.csv", editor
// Display the raw CSV file without importing it
editanything "$CA_INPUTS/market_value.csv", editor
/*
Optional: create a new Markdown project note.
Run this command only once. The new option correctly returns an error if
the file already exists.
*/
// editanything "$CA_ROOT/workflow_notes", new extension(md)
// =============================================================================
// PART 3 — MERGE ALL THREE CSV FILES WITH COMBINEALL
// =============================================================================
combineall using "$CA_MERGED", ///
cmethod(merge) ///
directory("$CA_INPUTS") ///
filetype(csv) ///
mtype(1:1) ///
mvars(firm_id year_id) ///
replace
// Display the stored results
return list
assert r(n_files) == 3
// =============================================================================
// PART 4 — LOAD AND VALIDATE THE MERGED PANEL
// =============================================================================
use "$CA_MERGED", clear
// Verify that the final merge produced complete matches
assert _merge == 3
drop _merge
// Recover conventional numeric panel identifiers
gen byte company = real(subinstr(firm_id, "F", "", 1))
gen int year = real(subinstr(year_id, "Y", "", 1))
order company year firm_id year_id invest mvalue kstock
sort company year
// Validate the merge
isid company year
assert _N == 200
assert !missing(invest, mvalue, kstock)
// Declare the panel
xtset company year
xtdescribe
summarize invest mvalue kstock
list company year invest mvalue kstock in 1/12, ///
sepby(company)
// Save the validated panel
compress
save "$CA_FINAL", replace
// =============================================================================
// PART 5 — ADVANCED HARMONIZATION EXAMPLE
// =============================================================================
/*
This example creates three annual files.
The variable is called invest_old in 1935 and 1936, but invest in 1937.
combineall will use a map file to harmonize the variable name before
appending the three releases.
*/
webuse grunfeld, clear
keep if inrange(year, 1935, 1937)
// -----------------------------------------------------------------------------
// Annual file for 1935
// -----------------------------------------------------------------------------
preserve
keep if year == 1935
rename invest invest_old
drop year
export delimited using ///
"$CA_ANNUAL/grunfeld_1935.csv", replace
restore
// -----------------------------------------------------------------------------
// Annual file for 1936
// -----------------------------------------------------------------------------
preserve
keep if year == 1936
rename invest invest_old
drop year
export delimited using ///
"$CA_ANNUAL/grunfeld_1936.csv", replace
restore
// -----------------------------------------------------------------------------
// Annual file for 1937
// -----------------------------------------------------------------------------
preserve
keep if year == 1937
drop year
export delimited using ///
"$CA_ANNUAL/grunfeld_1937.csv", replace
restore
// Verify the annual files
dir "$CA_ANNUAL/*.csv"
// =============================================================================
// PART 6 — CREATE THE VINTAGE-AWARE RENAME MAP
// =============================================================================
clear
input str20 oldname str20 newname firstyear lastyear
"invest_old" "invest" 1935 1936
end
list, noobs
export delimited using "$CA_MAP", replace
// =============================================================================
// PART 7 — APPEND AND HARMONIZE THE ANNUAL RELEASES
// =============================================================================
combineall using "$CA_HARMONIZED", ///
cmethod(append) ///
directory("$CA_ANNUAL") ///
filetype(csv) ///
map("$CA_MAP") ///
strict ///
replace
return list
assert r(n_files) == 3
assert r(n_missing) == 0
// Load and validate the harmonized panel
use "$CA_HARMONIZED", clear
sort company year
isid company year
assert _N == 30
assert inrange(year, 1935, 1937)
assert !missing(invest)
tabulate year
summarize invest mvalue kstock
// Display the provenance information
char list invest[source]
// =============================================================================
// PART 8 — LOAD THE COMPONENT FILES INTO SEPARATE FRAMES
// =============================================================================
/*
Warning: frames reset removes every dataset currently held in memory.
All important results have already been saved above.
*/
frames reset
// Reestablish the globals if this part is run in a new Stata session
global CA_ROOT "C:/Users/jamel/Dropbox/stata/combineall"
global CA_INPUTS "$CA_ROOT/grunfeld_merge_inputs"
global CA_FRAME_FINAL "$CA_ROOT/grunfeld_panel_from_frames.dta"
global CA_FRAMESET "$CA_ROOT/grunfeld_workflow"
// -----------------------------------------------------------------------------
// Investment frame
// -----------------------------------------------------------------------------
frame rename default investment
frame investment: import delimited using ///
"$CA_INPUTS/investment.csv", clear
// -----------------------------------------------------------------------------
// Market-value frame
// -----------------------------------------------------------------------------
frame create market
frame market: import delimited using ///
"$CA_INPUTS/market_value.csv", clear
// -----------------------------------------------------------------------------
// Capital-stock frame
// -----------------------------------------------------------------------------
frame create capital
frame capital: import delimited using ///
"$CA_INPUTS/capital_stock.csv", clear
// Display all datasets currently in memory
frames dir
// =============================================================================
// PART 9 — LINK THE THREE FRAMES
// =============================================================================
// Verify the identifiers within each frame
frame investment: isid firm_id year_id
frame market: isid firm_id year_id
frame capital: isid firm_id year_id
// Make investment the current frame
frame change investment
// Link the investment and market-value frames
frlink 1:1 firm_id year_id, frame(market)
frget mvalue, from(market)
// Link the investment and capital-stock frames
frlink 1:1 firm_id year_id, frame(capital)
frget kstock, from(capital)
// Validate the retrieved variables
assert !missing(invest, mvalue, kstock)
// Recover numeric panel identifiers
gen byte company = real(subinstr(firm_id, "F", "", 1))
gen int year = real(subinstr(year_id, "Y", "", 1))
order company year firm_id year_id invest mvalue kstock
sort company year
isid company year
assert _N == 200
xtset company year
xtdescribe
summarize invest mvalue kstock
list company year invest mvalue kstock in 1/12, ///
sepby(company)
// Save a clean standalone panel while preserving the live frame links
capture frame drop panel_clean
frame put company year firm_id year_id ///
invest mvalue kstock, into(panel_clean)
frame panel_clean: compress
frame panel_clean: save "$CA_FRAME_FINAL", replace
frame drop panel_clean
// Keep the three datasets available for inspection
frames dir
// Uncomment to open the final frame in the Data Editor
// browse company year invest mvalue kstock
// =============================================================================
// PART 10 — SAVE THE COLLECTION OF FRAMES
// =============================================================================
/*
frames save and the .dtas frameset format require Stata 18 or later.
*/
if c(stata_version) >= 18 {
frames save "$CA_FRAMESET", ///
frames(investment market capital) ///
replace
}
// =============================================================================
// END OF THE EXAMPLE
// =============================================================================
display as result ///
"Complete example finished successfully."
display as text ///
"Merged panel: $CA_FINAL"
display as text ///
"Harmonized panel: $CA_HARMONIZED"
display as text ///
"Frame-based panel: $CA_FRAME_FINAL"
frames dirOutput
. clear all
. set more off
.
. *-------------------------------------------------------------------------------
. * Portable project paths
. * The do-file may be launched from the root, code, data, or results directory
. *-------------------------------------------------------------------------------
.
. local here "`c(pwd)'"
. local here : subinstr local here "\" "/", all
.
. local last_folder = lower(substr( ///
> "`here'", ///
> strrpos("`here'", "/") + 1, ///
> . ///
> ))
.
. if inlist("`last_folder'", "code", "data", "results") {
. quietly cd ..
. }
.
. global main_path "`c(pwd)'"
. global code "${main_path}/code"
. global data "${main_path}/data"
. global results "${main_path}/results"
.
. cd "`c(pwd)'"
C:\Users\jamel\Dropbox\stata\combineall
.
. /*
> Execution dependencies
>
> - Run Part 0 first to install the packages and define the globals.
> - Parts 2 and 3 require the component CSV files created in Part 1.
> - Part 4 requires the merged file produced in Part 3.
> - Part 7 requires the annual CSV files and rename map created in
> Parts 5 and 6.
> - Part 8 requires the component CSV files created in Part 1.
> - Part 9 requires the frames created in Part 8.
> - Part 10 requires the linked frames produced in Parts 8 and 9.
> */
.
. // =============================================================================
. // PART 0 — INSTALL THE PACKAGES AND DEFINE THE PROJECT PATHS
. // =============================================================================
.
. // Install or update the packages
. ssc install combineall, replace
checking combineall consistency and verifying not already installed...
all files already exist and are up to date.
. ssc install editanything, replace
checking editanything consistency and verifying not already installed...
all files already exist and are up to date.
.
. which combineall
C:\Users\jamel\ado\plus\c\combineall.ado
*! combineall - Stata module to combine (append, merge, or joinby) or convert all files (.dta, ASCII, or Excel) in a directory
*! v2.0.1 30jul2026 Eric A. Booth, Sr Researcher, Texas 2036 <eric.a.booth@gmail.com>
*! Elizabeth Teas, Sr Research Scientist, Far Harbor, LLC <elizabeth@farharbor.com>
*! first released 2011 (v1.0.0, April 2011, Eric A. Booth)
*! v2.0.0 modernizes the 2011 engine (version 16 floor; import delimited and
*! import excel replace insheet) and grafts a harmonization layer, active
*! under cmethod(append): map(), year(), strict, char varname[source]
*! provenance, and a variable-by-year harmonization table.
. which editanything
C:\Users\jamel\ado\plus\e\editanything.ado
*! editanything v2.0.0 26may2026
*! Eric A. Booth, Sr Researcher, Texas 2036 (eric.a.booth@gmail.com)
*! Elizabeth Teas, Sr Research Scientist, Far Harbor, LLC (elizabeth@farharbor.com)
*! Open *any* text file (.ado, .sthlp, .hlp, .do, .mata, .md, .txt, .html,
*! .raw, .csv, .tsv, .R, .py, .json, .yaml, .toml, .xml, .css, .js, .sql,
*! .sh, .bat, .ini, .cfg, .log, .smcl, .tex, .bib, ...) from inside Stata.
*!
*! Default: opens in the Do-file Editor (like SSC's -adoedit-) but works on
*! any text-readable extension. Options let you preview in the Viewer, hand
*! the file to your OS default app, copy contents to the clipboard, create
*! a new empty file, or safely clone a base/updates ado into PERSONAL.
.
. // Main project directories and files
. global CA_ROOT "C:/Users/jamel/Dropbox/stata/combineall"
. global CA_INPUTS "$CA_ROOT/grunfeld_merge_inputs"
. global CA_MERGED "$CA_ROOT/grunfeld_merged_raw.dta"
. global CA_FINAL "$CA_ROOT/grunfeld_panel_final.dta"
.
. // Files used for the harmonization example
. global CA_ANNUAL "$CA_ROOT/grunfeld_annual_inputs"
. global CA_MAP "$CA_ROOT/grunfeld_rename_map.csv"
. global CA_HARMONIZED "$CA_ROOT/grunfeld_harmonized_1935_1937.dta"
.
. // Files produced by the frames example
. global CA_FRAME_FINAL "$CA_ROOT/grunfeld_panel_from_frames.dta"
. global CA_FRAMESET "$CA_ROOT/grunfeld_workflow"
.
. // Create the required directories
. capture mkdir "$CA_ROOT"
. capture mkdir "$CA_INPUTS"
. capture mkdir "$CA_ANNUAL"
.
. // Display the project paths
. display "$CA_ROOT"
C:/Users/jamel/Dropbox/stata/combineall
. display "$CA_INPUTS"
C:/Users/jamel/Dropbox/stata/combineall/grunfeld_merge_inputs
.
.
. // =============================================================================
. // PART 1 — CREATE THREE COMPONENT FILES FROM THE GRUNFELD PANEL
. // =============================================================================
.
. webuse grunfeld, clear
.
. describe
Contains data from https://www.stata-press.com/data/r19/grunfeld.dta
Observations: 200
Variables: 6 3 Mar 2024 20:27
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Variable Storage Display Value
name type format label Variable label
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
company byte %9.0g Company
year int %ty Year
invest float %9.0g Investment (current year)
mvalue float %9.0g Market value (prior year)
kstock float %9.0g Capital stock (prior year)
time byte %9.0g Time
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Sorted by: company year
. isid company year
.
. assert _N == 200
.
. /*
> combineall currently requires string identifiers for merge and joinby.
>
> The F and Y prefixes ensure that import delimited retains the identifiers
> as strings rather than interpreting them as numeric variables.
> */
.
. gen str3 firm_id = "F" + strtrim(strofreal(company))
. gen str5 year_id = "Y" + strtrim(strofreal(year))
.
. order company year firm_id year_id invest mvalue kstock
.
. isid firm_id year_id
.
. // -----------------------------------------------------------------------------
. // Investment file
. // -----------------------------------------------------------------------------
.
. preserve
.
. keep firm_id year_id invest
.
. export delimited using ///
> "$CA_INPUTS/investment.csv", replace
file C:/Users/jamel/Dropbox/stata/combineall/grunfeld_merge_inputs/investment.csv saved
.
. restore
.
. // -----------------------------------------------------------------------------
. // Market-value file
. // -----------------------------------------------------------------------------
.
. preserve
.
. keep firm_id year_id mvalue
.
. export delimited using ///
> "$CA_INPUTS/market_value.csv", replace
file C:/Users/jamel/Dropbox/stata/combineall/grunfeld_merge_inputs/market_value.csv saved
.
. restore
.
. // -----------------------------------------------------------------------------
. // Capital-stock file
. // -----------------------------------------------------------------------------
.
. preserve
.
. keep firm_id year_id kstock
.
. export delimited using ///
> "$CA_INPUTS/capital_stock.csv", replace
file C:/Users/jamel/Dropbox/stata/combineall/grunfeld_merge_inputs/capital_stock.csv saved
.
. restore
.
. // Verify the files
. dir "$CA_INPUTS/*.csv"
3.5k 8/04/26 0:02 capital_stock.csv
3.7k 8/04/26 0:02 investment.csv
3.6k 8/04/26 0:02 market_value.csv
.
.
. // =============================================================================
. // PART 2 — INSPECT THE FILES WITH EDITANYTHING
. // =============================================================================
.
. // Find the installed combineall ado-file without opening it
. editanything combineall, showpath
----------------------------------------------------------------------
file: C:/Users/jamel/ado/plus/c/combineall.ado
type: .ado size: 16,343 bytes
----------------------------------------------------------------------
.
. return list
scalars:
r(size) = 16343
macros:
r(basename) : "combineall.ado"
r(extension) : "ado"
r(file) : "C:/Users/jamel/ado/plus/c/combineall.ado"
.
. display "`r(file)'"
C:/Users/jamel/ado/plus/c/combineall.ado
. display "`r(basename)'"
combineall.ado
. display "`r(extension)'"
ado
. display r(size)
16343
.
. // Render the combineall help file read-only
. editanything combineall.sthlp, editor
----------------------------------------------------------------------
file: C:/Users/jamel/ado/plus/c/combineall.sthlp
type: .sthlp size: 22,107 bytes
----------------------------------------------------------------------
.
. // Display the raw CSV file without importing it
. editanything "$CA_INPUTS/investment.csv", editor
----------------------------------------------------------------------
file: C:/Users/jamel/Dropbox/stata/combineall/grunfeld_merge_inputs/investment.csv
type: .csv size: 3,759 bytes
----------------------------------------------------------------------
. // Display the raw CSV file without importing it
. editanything "$CA_INPUTS/capital_stock.csv", editor
----------------------------------------------------------------------
file: C:/Users/jamel/Dropbox/stata/combineall/grunfeld_merge_inputs/capital_stock.csv
type: .csv size: 3,625 bytes
----------------------------------------------------------------------
. // Display the raw CSV file without importing it
. editanything "$CA_INPUTS/market_value.csv", editor
----------------------------------------------------------------------
file: C:/Users/jamel/Dropbox/stata/combineall/grunfeld_merge_inputs/market_value.csv
type: .csv size: 3,676 bytes
----------------------------------------------------------------------
.
. /*
> Optional: create a new Markdown project note.
>
> Run this command only once. The new option correctly returns an error if
> the file already exists.
> */
.
. // editanything "$CA_ROOT/workflow_notes", new extension(md)
.
.
. // =============================================================================
. // PART 3 — MERGE ALL THREE CSV FILES WITH COMBINEALL
. // =============================================================================
.
. combineall using "$CA_MERGED", ///
> cmethod(merge) ///
> directory("$CA_INPUTS") ///
> filetype(csv) ///
> mtype(1:1) ///
> mvars(firm_id year_id) ///
> replace
Converted Files in Directory: C:/Users/jamel/Dropbox/stata/combineall/grunfeld_merge_inputs
Combined File: /Users/jamel/Dropbox/stata/combineall/grunfeld_merged_raw.dta"':C:/Users/jamel/Dropbox/stata/combineall/grunfeld_merged_raw.dta
.
. // Display the stored results
. return list
scalars:
r(n_files) = 3
macros:
r(output) : "C:/Users/jamel/Dropbox/stata/combineall/grunfeld_merged_raw.dta"
.
. assert r(n_files) == 3
.
.
. // =============================================================================
. // PART 4 — LOAD AND VALIDATE THE MERGED PANEL
. // =============================================================================
.
. use "$CA_MERGED", clear
.
. // Verify that the final merge produced complete matches
. assert _merge == 3
. drop _merge
.
. // Recover conventional numeric panel identifiers
. gen byte company = real(subinstr(firm_id, "F", "", 1))
. gen int year = real(subinstr(year_id, "Y", "", 1))
.
. order company year firm_id year_id invest mvalue kstock
. sort company year
.
. // Validate the merge
. isid company year
.
. assert _N == 200
. assert !missing(invest, mvalue, kstock)
.
. // Declare the panel
. xtset company year
Panel variable: company (strongly balanced)
Time variable: year, 1935 to 1954
Delta: 1 unit
.
. xtdescribe
company: 1, 2, ..., 10 n = 10
year: 1935, 1936, ..., 1954 T = 20
Delta(year) = 1 unit
Span(year) = 20 periods
(company*year uniquely identifies each observation)
Distribution of T_i: min 5% 25% 50% 75% 95% max
20 20 20 20 20 20 20
Freq. Percent Cum. | Pattern
---------------------------+----------------------
10 100.00 100.00 | 11111111111111111111
---------------------------+----------------------
10 100.00 | XXXXXXXXXXXXXXXXXXXX
.
. summarize invest mvalue kstock
Variable | Obs Mean Std. dev. Min Max
-------------+---------------------------------------------------------
invest | 200 145.9583 216.8753 .93 1486.7
mvalue | 200 1081.681 1314.47 58.12 6241.7
kstock | 200 276.0172 301.1039 .8 2226.3
.
. list company year invest mvalue kstock in 1/12, ///
> sepby(company)
+----------------------------------------------------+
| company year invest mvalue kstock |
|----------------------------------------------------|
1. | 1 1935 317.60001 3078.5 2.8 |
2. | 1 1936 391.79999 4661.7002 52.599998 |
3. | 1 1937 410.60001 5387.1001 156.89999 |
4. | 1 1938 257.70001 2792.2 209.2 |
5. | 1 1939 330.79999 4313.2002 203.39999 |
6. | 1 1940 461.20001 4643.8999 207.2 |
7. | 1 1941 512 4551.2002 255.2 |
8. | 1 1942 448 3244.1001 303.70001 |
9. | 1 1943 499.60001 4053.7 264.10001 |
10. | 1 1944 547.5 4379.2998 201.60001 |
11. | 1 1945 561.20001 4840.8999 265 |
12. | 1 1946 688.09998 4900.8999 402.20001 |
+----------------------------------------------------+
.
. // Save the validated panel
. compress
(0 bytes saved)
.
. save "$CA_FINAL", replace
file C:/Users/jamel/Dropbox/stata/combineall/grunfeld_panel_final.dta saved
.
.
. // =============================================================================
. // PART 5 — ADVANCED HARMONIZATION EXAMPLE
. // =============================================================================
.
. /*
> This example creates three annual files.
>
> The variable is called invest_old in 1935 and 1936, but invest in 1937.
> combineall will use a map file to harmonize the variable name before
> appending the three releases.
> */
.
. webuse grunfeld, clear
.
. keep if inrange(year, 1935, 1937)
(170 observations deleted)
.
. // -----------------------------------------------------------------------------
. // Annual file for 1935
. // -----------------------------------------------------------------------------
.
. preserve
.
. keep if year == 1935
(20 observations deleted)
.
. rename invest invest_old
.
. drop year
.
. export delimited using ///
> "$CA_ANNUAL/grunfeld_1935.csv", replace
file C:/Users/jamel/Dropbox/stata/combineall/grunfeld_annual_inputs/grunfeld_1935.csv saved
.
. restore
.
. // -----------------------------------------------------------------------------
. // Annual file for 1936
. // -----------------------------------------------------------------------------
.
. preserve
.
. keep if year == 1936
(20 observations deleted)
.
. rename invest invest_old
.
. drop year
.
. export delimited using ///
> "$CA_ANNUAL/grunfeld_1936.csv", replace
file C:/Users/jamel/Dropbox/stata/combineall/grunfeld_annual_inputs/grunfeld_1936.csv saved
.
. restore
.
. // -----------------------------------------------------------------------------
. // Annual file for 1937
. // -----------------------------------------------------------------------------
.
. preserve
.
. keep if year == 1937
(20 observations deleted)
.
. drop year
.
. export delimited using ///
> "$CA_ANNUAL/grunfeld_1937.csv", replace
file C:/Users/jamel/Dropbox/stata/combineall/grunfeld_annual_inputs/grunfeld_1937.csv saved
.
. restore
.
. // Verify the annual files
. dir "$CA_ANNUAL/*.csv"
0.3k 8/04/26 0:02 grunfeld_1935.csv
0.3k 8/04/26 0:02 grunfeld_1936.csv
0.3k 8/04/26 0:02 grunfeld_1937.csv
.
.
. // =============================================================================
. // PART 6 — CREATE THE VINTAGE-AWARE RENAME MAP
. // =============================================================================
.
. clear
.
. input str20 oldname str20 newname firstyear lastyear
oldname newname firstyear lastyear
1. "invest_old" "invest" 1935 1936
2. end
.
. list, noobs
+--------------------------------------------+
| oldname newname firsty~r lastyear |
|--------------------------------------------|
| invest_old invest 1935 1936 |
+--------------------------------------------+
.
. export delimited using "$CA_MAP", replace
file C:/Users/jamel/Dropbox/stata/combineall/grunfeld_rename_map.csv saved
.
.
. // =============================================================================
. // PART 7 — APPEND AND HARMONIZE THE ANNUAL RELEASES
. // =============================================================================
.
. combineall using "$CA_HARMONIZED", ///
> cmethod(append) ///
> directory("$CA_ANNUAL") ///
> filetype(csv) ///
> map("$CA_MAP") ///
> strict ///
> replace
Converted Files in Directory: C:/Users/jamel/Dropbox/stata/combineall/grunfeld_annual_inputs
Combined File: /Users/jamel/Dropbox/stata/combineall/grunfeld_harmonized_1935_1937.dta"':C:/Users/jamel/Dropbox/stata/combineall/grunfeld_harmonized_1935_1937.dta
Harmonization table (X = data present in that year):
variable 1935 1936 1937
------------------------------------------
company X X X
invest X X X
mvalue X X X
kstock X X X
time X X X
combineall: 3 file(s), 30 observations, 6 variables, years 1935 1936 1937
.
. return list
scalars:
r(n_missing) = 0
r(n_vars) = 6
r(n_files) = 3
macros:
r(years) : "1935 1936 1937"
r(output) : "C:/Users/jamel/Dropbox/stata/combineall/grunfeld_harmonized_1935_1937.dta"
.
. assert r(n_files) == 3
. assert r(n_missing) == 0
.
. // Load and validate the harmonized panel
. use "$CA_HARMONIZED", clear
.
. sort company year
.
. isid company year
.
. assert _N == 30
. assert inrange(year, 1935, 1937)
. assert !missing(invest)
.
. tabulate year
year | Freq. Percent Cum.
------------+-----------------------------------
1935 | 10 33.33 33.33
1936 | 10 33.33 66.67
1937 | 10 33.33 100.00
------------+-----------------------------------
Total | 30 100.00
.
. summarize invest mvalue kstock
Variable | Obs Mean Std. dev. Min Max
-------------+---------------------------------------------------------
invest | 30 98.94467 138.7963 2 469.9
mvalue | 30 1046.445 1390.531 70.91 5387.1
kstock | 30 79.796 74.1797 .8 236
.
. // Display the provenance information
. char list invest[source]
invest[source]: invest_old (grunfeld_1935.csv, 1935); invest_old (grunfeld_1936.csv, 1936)
.
.
. // =============================================================================
. // PART 8 — LOAD THE COMPONENT FILES INTO SEPARATE FRAMES
. // =============================================================================
.
. /*
> Warning: frames reset removes every dataset currently held in memory.
> All important results have already been saved above.
> */
.
. frames reset
.
. // Reestablish the globals if this part is run in a new Stata session
. global CA_ROOT "C:/Users/jamel/Dropbox/stata/combineall"
. global CA_INPUTS "$CA_ROOT/grunfeld_merge_inputs"
. global CA_FRAME_FINAL "$CA_ROOT/grunfeld_panel_from_frames.dta"
. global CA_FRAMESET "$CA_ROOT/grunfeld_workflow"
.
. // -----------------------------------------------------------------------------
. // Investment frame
. // -----------------------------------------------------------------------------
.
. frame rename default investment
.
. frame investment: import delimited using ///
> "$CA_INPUTS/investment.csv", clear
(3 vars, 200 obs)
.
. // -----------------------------------------------------------------------------
. // Market-value frame
. // -----------------------------------------------------------------------------
.
. frame create market
.
. frame market: import delimited using ///
> "$CA_INPUTS/market_value.csv", clear
(3 vars, 200 obs)
.
. // -----------------------------------------------------------------------------
. // Capital-stock frame
. // -----------------------------------------------------------------------------
.
. frame create capital
.
. frame capital: import delimited using ///
> "$CA_INPUTS/capital_stock.csv", clear
(3 vars, 200 obs)
.
. // Display all datasets currently in memory
. frames dir
* capital 200 x 3
* investment 200 x 3
* market 200 x 3
Note: Frames marked with * contain unsaved data.
.
.
. // =============================================================================
. // PART 9 — LINK THE THREE FRAMES
. // =============================================================================
.
. // Verify the identifiers within each frame
. frame investment: isid firm_id year_id
. frame market: isid firm_id year_id
. frame capital: isid firm_id year_id
.
. // Make investment the current frame
. frame change investment
.
. // Link the investment and market-value frames
. frlink 1:1 firm_id year_id, frame(market)
(all observations in frame investment matched)
.
. frget mvalue, from(market)
(1 variable copied from linked frame)
.
. // Link the investment and capital-stock frames
. frlink 1:1 firm_id year_id, frame(capital)
(all observations in frame investment matched)
.
. frget kstock, from(capital)
(1 variable copied from linked frame)
.
. // Validate the retrieved variables
. assert !missing(invest, mvalue, kstock)
.
. // Recover numeric panel identifiers
. gen byte company = real(subinstr(firm_id, "F", "", 1))
. gen int year = real(subinstr(year_id, "Y", "", 1))
.
. order company year firm_id year_id invest mvalue kstock
. sort company year
.
. isid company year
.
. assert _N == 200
.
. xtset company year
Panel variable: company (strongly balanced)
Time variable: year, 1935 to 1954
Delta: 1 unit
.
. xtdescribe
company: 1, 2, ..., 10 n = 10
year: 1935, 1936, ..., 1954 T = 20
Delta(year) = 1 unit
Span(year) = 20 periods
(company*year uniquely identifies each observation)
Distribution of T_i: min 5% 25% 50% 75% 95% max
20 20 20 20 20 20 20
Freq. Percent Cum. | Pattern
---------------------------+----------------------
10 100.00 100.00 | 11111111111111111111
---------------------------+----------------------
10 100.00 | XXXXXXXXXXXXXXXXXXXX
.
. summarize invest mvalue kstock
Variable | Obs Mean Std. dev. Min Max
-------------+---------------------------------------------------------
invest | 200 145.9583 216.8753 .93 1486.7
mvalue | 200 1081.681 1314.47 58.12 6241.7
kstock | 200 276.0172 301.1039 .8 2226.3
.
. list company year invest mvalue kstock in 1/12, ///
> sepby(company)
+-------------------------------------------+
| company year invest mvalue kstock |
|-------------------------------------------|
1. | 1 1935 317.6 3078.5 2.8 |
2. | 1 1936 391.8 4661.7 52.6 |
3. | 1 1937 410.6 5387.1 156.9 |
4. | 1 1938 257.7 2792.2 209.2 |
5. | 1 1939 330.8 4313.2 203.4 |
6. | 1 1940 461.2 4643.9 207.2 |
7. | 1 1941 512 4551.2 255.2 |
8. | 1 1942 448 3244.1 303.7 |
9. | 1 1943 499.6 4053.7 264.1 |
10. | 1 1944 547.5 4379.3 201.6 |
11. | 1 1945 561.2 4840.9 265 |
12. | 1 1946 688.1 4900.9 402.2 |
+-------------------------------------------+
.
. // Save a clean standalone panel while preserving the live frame links
. capture frame drop panel_clean
.
. frame put company year firm_id year_id ///
> invest mvalue kstock, into(panel_clean)
.
. frame panel_clean: compress
(0 bytes saved)
. frame panel_clean: save "$CA_FRAME_FINAL", replace
file C:/Users/jamel/Dropbox/stata/combineall/grunfeld_panel_from_frames.dta saved
.
. frame drop panel_clean
.
. // Keep the three datasets available for inspection
. frames dir
* capital 200 x 3
* investment 200 x 9
* market 200 x 3
Note: Frames marked with * contain unsaved data.
.
. // Uncomment to open the final frame in the Data Editor
. // browse company year invest mvalue kstock
.
.
. // =============================================================================
. // PART 10 — SAVE THE COLLECTION OF FRAMES
. // =============================================================================
.
. /*
> frames save and the .dtas frameset format require Stata 18 or later.
> */
.
. if c(stata_version) >= 18 {
.
. frames save "$CA_FRAMESET", ///
> frames(investment market capital) ///
> replace
file C:/Users/jamel/Dropbox/stata/combineall/grunfeld_workflow.dtas saved
. }
.
.
. // =============================================================================
. // END OF THE EXAMPLE
. // =============================================================================
.
. display as result ///
> "Complete example finished successfully."
Complete example finished successfully.
.
. display as text ///
> "Merged panel: $CA_FINAL"
Merged panel: C:/Users/jamel/Dropbox/stata/combineall/grunfeld_panel_final.dta
.
. display as text ///
> "Harmonized panel: $CA_HARMONIZED"
Harmonized panel: C:/Users/jamel/Dropbox/stata/combineall/grunfeld_harmonized_1935_1937.dta
.
. display as text ///
> "Frame-based panel: $CA_FRAME_FINAL"
Frame-based panel: C:/Users/jamel/Dropbox/stata/combineall/grunfeld_panel_from_frames.dta
.
. frames dir
* capital 200 x 3
* investment 200 x 9
* market 200 x 3
Note: Frames marked with * contain unsaved data.
.
end of do-file
.