medicaid_utils.common_utils package¶

Submodules¶

medicaid_utils.common_utils.dataframe_utils module¶

medicaid_utils.common_utils.dataframe_utils.convert_ddcols_to_datetime(df: DataFrame, lst_col: List[str]) → DataFrame[source]¶: Convert list of columns specified in a dataframe to datetime type :param pandas_df df: dataframe :param list(str) lst_col: list of column names :rtype: None

medicaid_utils.common_utils.dataframe_utils.copy_ddcols(df: DataFrame, lst_col: List[str], lst_new_names: List[str]) → DataFrame[source]¶

medicaid_utils.common_utils.dataframe_utils.export(df: DataFrame, pq_engine: str, output_filename: str, pq_location: str, _csv_location: str, lst_datetime_col: List[str], is_dask: bool = True, n_rows: int = -1, do_csv: bool = True, df_schema: DataFrame | None = None, logger_name: str = 'Dataframe utils', rewrite: bool = False, do_parquet: bool = True) → None[source]¶: Exports a Dask DataFrame to parquet and/or CSV.

medicaid_utils.common_utils.dataframe_utils.fix_index(df: DataFrame, index_name: str, drop_column: bool = True) → DataFrame[source]¶

medicaid_utils.common_utils.dataframe_utils.get_first_day_gap(df: DataFrame, index_col: str, time_col: str, start_date_col: str, threshold: int) → DataFrame[source]¶

medicaid_utils.common_utils.dataframe_utils.get_reduced_column_names(multiidx_df_columns: MultiIndex, combine_levels: bool = False) → List[str][source]¶

medicaid_utils.common_utils.dataframe_utils.prepare_dtypes_for_csv(df_temp: DataFrame, df_schema: DataFrame) → DataFrame[source]¶

medicaid_utils.common_utils.dataframe_utils.safe_convert_int_to_str(df: DataFrame, lst_col: List[str]) → DataFrame[source]¶

medicaid_utils.common_utils.dataframe_utils.sas_to_pandas(filename: str) → DataFrame[source]¶

medicaid_utils.common_utils.dataframe_utils.toggle_datetime_string(df: DataFrame, lst_datetime_col: List[str], to_string: bool = True) → DataFrame[source]¶

Toggles date columns in the passed dataframe to string/ datetime types.

Parameters:

df (dd.DataFrame) – Dask dataframe
lst_datetime_col (list of str) – List of datetime column names to toggle
to_string (bool, default=True) – True to convert to string, False to convert to datetime

Return type:

dd.DataFrame

medicaid_utils.common_utils.links module¶

medicaid_utils.common_utils.links.get_max_parquet_loc(root: str, claim_type: str, state: str, year: int | str) → str[source]¶

medicaid_utils.common_utils.links.get_taf_parquet_loc(root: str, claim_type: str, state: str, year: int | str) → Dict[str, str][source]¶

medicaid_utils.common_utils.recipes module¶

medicaid_utils.common_utils.recipes.convert_to_int_str(x: Any) → str | Any[source]¶

medicaid_utils.common_utils.recipes.is_number(x: Any) → bool[source]¶

medicaid_utils.common_utils.recipes.log_assert(bool_: bool, message: str = '', logger: Logger | None = None, logger_name: str = '', verbose: bool = False) → None[source]¶: Use this as a replacement for assert if you want the failing of the assert statement to be logged.

medicaid_utils.common_utils.recipes.remove_ignore_if_not_exists(filename: str) → None[source]¶

medicaid_utils.common_utils.recipes.remove_tail_dot_zeros(a: str) → str[source]¶

medicaid_utils.common_utils.recipes.setup_logger(logger_name: str, log_file: str, level: int = 20) → None[source]¶

medicaid_utils.common_utils.stats_utils module¶

medicaid_utils.common_utils.stats_utils.color_positive_green(x: DataFrame) → DataFrame[source]¶

medicaid_utils.common_utils.stats_utils.compute_contingency_table(pdf: DataFrame, lst_states: List[str], lst_metrics: List[str], lst_count_metrics: List[str], output_fname: str, pop_col_name: str = 'gt_50pc_hrsa_fqhc', dct_labels: Dict[int, str] | None = None, state_col_name: str = 'STATE_CD') → pd.io.formats.style.Styler[source]¶

medicaid_utils.common_utils.stats_utils.compute_descriptives(pdf: DataFrame, lst_states: List[str], lst_metrics: List[str], output_fname: str, state_col_name: str = 'STATE_CD') → DataFrame[source]¶

medicaid_utils.common_utils.stats_utils.compute_missing_stats(df: DataFrame, output_fname: str, state_col_name: str = 'STATE_CD') → DataFrame[source]¶

medicaid_utils.common_utils.stats_utils.compute_t_stats(pdf: DataFrame, lst_states: List[str], lst_metrics: List[str], output_fname: str, pop_col_name: str = 'gt_50pc_hrsa_fqhc', dct_labels: Dict[int, str] | None = None, state_col_name: str = 'STATE_CD') → pd.io.formats.style.Styler[source]¶

medicaid_utils.common_utils.stats_utils.cramers_corrected_stat(confusion_matrix: DataFrame) → float[source]¶: calculate Cramers V statistic for categorial-categorial association. uses correction from Bergsma and Wicher, Journal of the Korean Statistical Society 42 (2013): 323-328

medicaid_utils.common_utils.stats_utils.get_cont_table_statewise(pdf_included: DataFrame, lst_metrics: List[str], pop_col_name: str, lst_count_metrics: List[str], dct_labels: Dict[int, str], lst_st: List[str], state_col_name: str) → DataFrame[source]¶

medicaid_utils.common_utils.stats_utils.get_contingency_table(pdf_dataset: DataFrame, lst_categorical_metrics: List[str], pop_col_name: str, lst_numeric_col_to_binarize: List[str], dct_labels: Dict[int, str]) → Tuple[DataFrame, DataFrame][source]¶

medicaid_utils.common_utils.stats_utils.get_covar_plots(pdf: DataFrame, lst_covar: List[str], lst_hist_covar: List[str], cut_outliers: bool = False) → Any[source]¶

medicaid_utils.common_utils.stats_utils.get_descriptives(pdf: DataFrame, lst_st: List[str], lst_col: List[str], state_col_name: str) → DataFrame[source]¶

medicaid_utils.common_utils.stats_utils.get_missingness_stats(df: DataFrame, outputfname: str) → DataFrame[source]¶

medicaid_utils.common_utils.stats_utils.get_phi(pdf_x: DataFrame) → float[source]¶

medicaid_utils.common_utils.stats_utils.get_ranksum_table(pdf_dataset: DataFrame, lst_metrics: List[str], pop_col_name: str, dct_labels: Dict[int, str]) → Tuple[DataFrame, DataFrame][source]¶

medicaid_utils.common_utils.stats_utils.get_ranksum_table_statewise(pdf_included: DataFrame, lst_metrics: List[str], pop_col_name: str, dct_labels: Dict[int, str], lst_st: List[str], state_col_name: str) → DataFrame[source]¶

medicaid_utils.common_utils.stats_utils.get_utilisation_histograms(pdf: DataFrame, lst_covar: List[str]) → Any[source]¶

medicaid_utils.common_utils.usps_address module¶

This script shows an example of using requests and the USPS Address Information API. In order to use this, you must first register so you can get your USERID. Your ID must be in the environment variable USPS_USERID. For information on the API see here <https://www.usps.com/business/web-tools-apis/address-information-api.htm>_

class medicaid_utils.common_utils.usps_address.AddressStandardizationWebTool(street, city, state, name=None, suite=None, zip5=None, zip4=None, userid='')[source]¶

Bases: USPSShippingAPI

Object to get a standardized USPS Address.

api = 'Verify'¶

get_standardized_address()[source]¶: Returns a standardized format of the object’s address.

class medicaid_utils.common_utils.usps_address.USPSAddress(name='', suite='', street='', city='', state='', zip5='', zip4='')[source]¶

Bases: object

Representation of an United States Postal Service address.

original()[source]¶: Return the non-standardized address format

standardized()[source]¶: Return the standardized address format

zipcode()[source]¶: Returns the zipcode based on whether or not zip4 is used.

class medicaid_utils.common_utils.usps_address.USPSShippingAPI(api, userid='')[source]¶

Bases: object

Representation of the USPS Shipping API https://www.usps.com/business/web-tools-apis/address-information-api.htm

send_request()[source]¶: Send the request and return the XML response.

url = 'https://production.shippingapis.com/ShippingAPI.dll'¶

medicaid_utils.common_utils.usps_address.dump_xml(raw_xml)[source]¶: Return a string representation of XML with proper intendation

medicaid_utils.common_utils.usps_address.get_text(root, xpath)[source]¶: Return the text of the XPath element, or None if the element was not found.