Hướng dẫn pd read_excel skip columns
I know beforehand what columns I don't need from an excel file and I'd like to avoid them when reading the file to improve the performance. Something like this:
There is nothing related to this in the documentation. is there any workaround for this?
Aran-Fey 36.6k11 gold badges96 silver badges141 bronze badges asked Apr 5, 2018 at 16:32
5 If your version of pandas allows (check first if you can pass a function to usecols), I would try something like:
This should skip all columns without header names. You could substitute 'Unnamed' with a list of column names you do not want. answered May 22, 2019 at 8:28
MarMatMarMat 6545 silver badges12 bronze badges 2 You can use the
following technique. Let the columns we don't want(want to skip) are 2 5 8, then find all reamining columns we DO WANT TO KEEP as
and then we can use those remaining columns(which we DO WANT TO KEEP) using
answered Apr 5, 2018 at 17:14
1 Read an Excel file into a pandas DataFrame. Supports xls, xlsx, xlsm, xlsb, odf, ods and odt file extensions read from a local filesystem or URL. Supports an option to read a single sheet or a list of sheets. Parametersiostr, bytes, ExcelFile, xlrd.Book, path object, or file-like objectAny valid string path is acceptable. The string could be a URL. Valid URL schemes include http, ftp, s3,
and file. For file URLs, a host is expected. A local file could be: If you want to pass in a path object, pandas accepts any By file-like object, we refer to objects with a Strings are used for sheet names. Integers are used in zero-indexed sheet positions (chart sheets do not count as a sheet position). Lists of strings/integers are used to request multiple sheets. Specify None to get all worksheets. Available cases:
Row (0-indexed) to use for the column labels of the parsed DataFrame. If a list of integers is passed those row positions will be combined into a List of column names to use. If file contains no header row, then you should explicitly pass header=None. index_colint, list of int, default NoneColumn (0-indexed) to use as the row labels of the DataFrame. Pass None if there is no such column. If a list is passed, those columns will be combined into a Missing values will be forward filled to allow roundtripping with
Returns a subset of the columns according to behavior above. squeezebool, default FalseIf the parsed data only contains one column then return a Series. Deprecated since version 1.4.0: Append Data type for data or columns. E.g. {‘a’: np.float64, ‘b’: np.int32} Use object to preserve data as stored in Excel and not interpret dtype. If converters are specified, they will be applied INSTEAD of dtype conversion. enginestr, default NoneIf io is not a buffer or path, this must be set to identify io. Supported engines: “xlrd”, “openpyxl”, “odf”, “pyxlsb”. Engine compatibility :
Changed in version 1.2.0: The engine xlrd now only supports old-style
Dict of functions for converting values in certain columns. Keys can either be integers or column labels, values are functions that take one input argument, the Excel cell content, and return the transformed content. true_valueslist, default NoneValues to consider as True. false_valueslist, default NoneValues to consider as False. skiprowslist-like, int, or callable, optionalLine numbers to skip (0-indexed) or number of lines to skip (int) at the start of the file. If callable, the callable function will be evaluated against the row indices, returning True if the row should be skipped and False otherwise. An example of a valid callable argument would be Number of rows to parse. na_valuesscalar, str, list-like, or dict, default NoneAdditional strings to recognize as NA/NaN. If dict passed, specific per-column NA values. By default the following values are interpreted as NaN: ‘’, ‘#N/A’, ‘#N/A N/A’, ‘#NA’, ‘-1.#IND’, ‘-1.#QNAN’, ‘-NaN’, ‘-nan’, ‘1.#IND’, ‘1.#QNAN’, ‘ Whether or not to include the default NaN values when parsing the data. Depending on whether na_values is passed in, the behavior is as follows:
Note that if na_filter is passed in as False, the keep_default_na and na_values parameters will be ignored. na_filterbool, default TrueDetect missing value markers (empty strings and the value of na_values). In data without any NAs, passing na_filter=False can improve the performance of reading a large file. verbosebool, default FalseIndicate number of NA values placed in non-numeric columns. parse_datesbool, list-like, or dict, default FalseThe behavior is as follows:
If a column or index contains an unparsable date, the entire column or index will be returned unaltered as an object
data type. If you don`t want to parse some cells as date just change their type in Excel to “Text”. For non-standard datetime parsing, use Note: A fast-path exists for iso8601-formatted dates. date_parserfunction, optionalFunction to use for converting a sequence of string columns to an array of datetime instances. The default uses Thousands separator for parsing string columns to numeric. Note that this parameter is only necessary for columns stored as TEXT in Excel, any numeric columns will automatically be parsed, regardless of display format. decimalstr, default ‘.’Character to recognize as decimal point for parsing string columns to numeric. Note that this parameter is only necessary for columns stored as TEXT in Excel, any numeric columns will automatically be parsed, regardless of display format.(e.g. use ‘,’ for European data). New in version 1.4.0. commentstr, default NoneComments out remainder of line. Pass a character or characters to this argument to indicate comments in the input file. Any data between the comment string and the end of the current line is ignored. skipfooterint, default 0Rows at the end to skip (0-indexed). convert_floatbool, default TrueConvert integral floats to int (i.e., 1.0 –> 1). If False, all numeric data will be read in as floats: Excel stores all numbers as floats internally. Deprecated since version 1.3.0: convert_float will be removed in a future version mangle_dupe_colsbool, default TrueDuplicate columns will be specified as ‘X’, ‘X.1’, …’X.N’, rather than ‘X’…’X’. Passing in False will cause data to be overwritten if there are duplicate names in the columns. Deprecated since version 1.5.0: Not implemented, and a new argument to specify the pattern for the names of duplicated columns will be added instead storage_optionsdict, optionalExtra options that make sense for a particular storage connection, e.g. host, port, username, password, etc. For
HTTP(S) URLs the key-value pairs are forwarded to New in version 1.2.0. Returns DataFrame or dict of DataFramesDataFrame from the passed in Excel file. See notes in sheet_name argument for more information on when a dict of DataFrames is returned. See also DataFrame.to_excel Write DataFrame to an Excel file. DataFrame.to_csv Write DataFrame to a comma-separated values (csv) file. read_csv Read a comma-separated values (csv) file into DataFrame. read_fwf Read a table of fixed-width formatted lines into DataFrame. Examples The file can be read using the file name as string or an open file object: >>> pd.read_excel('tmp.xlsx', index_col=0) Name Value 0 string1 1 1 string2 2 2 #Comment 3 >>> pd.read_excel(open('tmp.xlsx', 'rb'), ... sheet_name='Sheet3') Unnamed: 0 Name Value 0 0 string1 1 1 1 string2 2 2 2 #Comment 3 Index and header can be specified via the index_col and header arguments >>> pd.read_excel('tmp.xlsx', index_col=None, header=None) 0 1 2 0 NaN Name Value 1 0.0 string1 1 2 1.0 string2 2 3 2.0 #Comment 3 Column types are inferred but can be explicitly specified >>> pd.read_excel('tmp.xlsx', index_col=0, ... dtype={'Name': str, 'Value': float}) Name Value 0 string1 1.0 1 string2 2.0 2 #Comment 3.0 True, False, and NA values, and thousands separators have defaults, but can be explicitly specified, too. Supply the values you would like as strings or lists of strings! >>> pd.read_excel('tmp.xlsx', index_col=0, ... na_values=['string1', 'string2']) Name Value 0 NaN 1 1 NaN 2 2 #Comment 3 Comment lines in the excel input file can be skipped using the comment kwarg >>> pd.read_excel('tmp.xlsx', index_col=0, comment='#') Name Value 0 string1 1.0 1 string2 2.0 2 None NaN |