OwlCyberSecurity - MANAGER
Edit File: _inputstream.cpython-39.pyc
a ��Rea~ � @ s� d dl mZmZmZ d dlmZ d dlmZmZ d dl Z d dl Z d dl m Z m Z d dlmZ ddlmZmZmZmZ dd lmZ dd lmZ ed d � eD ��Zed d � eD ��Zedd � eD ��Zeeddg�B Z dZ ej �r e d dkr�e � d�dks�J �e � e dd� e!d� d �Z"n e � e �Z"h d�Z#e � d�Z$i Z%G dd� de&�Z'dd� Z(G dd � d e&�Z)G d d � d e)�Z*G d d � d e+�Z,G d!d"� d"e&�Z-G d#d$� d$e&�Z.d%d&� Z/dS )'� )�absolute_import�division�unicode_literals)� text_type)� http_client�urllibN)�BytesIO�StringIO)� webencodings� )�EOF�spaceCharacters� asciiLetters�asciiUppercase)�_ReparseException)�_utilsc C s g | ]}|� d ��qS ��ascii��encode��.0�item� r ��/builddir/build/BUILDROOT/alt-python39-pip-21.3.1-2.el8.x86_64/opt/alt/python39/lib/python3.9/site-packages/pip/_vendor/html5lib/_inputstream.py� <listcomp> � r c C s g | ]}|� d ��qS r r r r r r r r c C s g | ]}|� d ��qS r r r r r r r r � >� <u� [- - --]����]z"\uD800-\uDFFF"> i�� i�� i�� i�� i�� i�� i�� i�� i�� i�� i�� i�� i�� i�� i�� i�� i�� i�� i�� i�� i�� i�� i�� i�� i�� i�� i�� i�� i�� i�� i�� i�� z[ - -/:-@\[-`{-~]c @ sH e Zd ZdZdd� Zdd� Zdd� Zdd � Zd d � Zd d � Z dd� Z dS )�BufferedStreamz�Buffering for streams that do not have buffering of their own The buffer is implemented as a list of chunks on the assumption that joining many strings will be slow since it is O(n**2) c C s || _ g | _ddg| _d S )Nr r )�stream�buffer�position)�selfr" r r r �__init__: s zBufferedStream.__init__c C s< d}| j d | jd � D ]}|t|�7 }q|| jd 7 }|S �Nr r )r# r$ �len)r% �pos�chunkr r r �tell? s zBufferedStream.tellc C sT || � � ksJ �|}d}t| j| �|k rF|t| j| �8 }|d7 }q||g| _d S r' )�_bufferedBytesr( r# r$ )r% r) �offset�ir r r �seekF s zBufferedStream.seekc C sT | j s| �|�S | jd t| j �krF| jd t| j d �krF| �|�S | �|�S d S )Nr r r )r# � _readStreamr$ r( �_readFromBuffer�r% �bytesr r r �readO s � zBufferedStream.readc C s t dd� | jD ��S )Nc S s g | ] }t |��qS r )r( r r r r r Y r z1BufferedStream._bufferedBytes.<locals>.<listcomp>)�sumr# �r% r r r r, X s z BufferedStream._bufferedBytesc C s< | j �|�}| j�|� | jd d7 < t|�| jd< |S r' )r" r4 r# �appendr$ r( )r% r3 �datar r r r0 [ s zBufferedStream._readStreamc C s� |}g }| j d }| j d }|t| j�k r�|dkr�|dks>J �| j| }|t|�| krl|}||| g| _ n"t|�| }|t|�g| _ |d7 }|�|||| � � ||8 }d}q |r�|�| �|�� d�|�S )Nr r r )r$ r( r# r7 r0 �join)r% r3 �remainingBytes�rv� bufferIndex� bufferOffset� bufferedData� bytesToReadr r r r1 b s& z BufferedStream._readFromBufferN) �__name__� __module__� __qualname__�__doc__r& r+ r/ r4 r, r0 r1 r r r r r! 3 s r! c K s� t | tj�s(t | tjj�r.t | jtj�r.d}n&t| d�rJt | �d�t �}n t | t �}|r�dd� |D �}|rvt d| ��t | fi |��S t | fi |��S d S )NFr4 r c S s g | ]}|� d �r|�qS )� _encoding)�endswith)r �xr r r r � r z#HTMLInputStream.<locals>.<listcomp>z3Cannot set an encoding with a unicode input, set %r) � isinstancer � HTTPResponser �response�addbase�fp�hasattrr4 r � TypeError�HTMLUnicodeInputStream�HTMLBinaryInputStream)�source�kwargs� isUnicode� encodingsr r r �HTMLInputStream} s � � rT c @ sp e Zd ZdZdZdd� Zdd� Zdd� Zd d � Zd d � Z d d� Z ddd�Z dd� Z dd� Z d dd�Zdd� ZdS ) rN ��Provides a unicode stream of characters to the HTMLTokenizer. This class takes care of character encoding and removing or replacing incorrect byte-sequences and also provides column and line tracking. i ( c C sZ t jsd| _n td�dkr$| j| _n| j| _dg| _td�df| _| � |�| _ | � � dS )� Initialises the HTMLInputStream. HTMLInputStream(source, [encoding]) -> Normalized stream from source for use by html5lib. source can be either a file-object, local filename or a string. The optional encoding parameter must be a string that indicates the encoding. If specified, that encoding will be used, regardless of any BOM or later declaration (such as in a meta element) Nu r r �utf-8�certain) r �supports_lone_surrogates�reportCharacterErrorsr( �characterErrorsUCS4�characterErrorsUCS2�newLines�lookupEncoding� charEncoding� openStream� dataStream�reset)r% rP r r r r& � s z HTMLUnicodeInputStream.__init__c C s. d| _ d| _d| _g | _d| _d| _d | _d S )N� r )r* � chunkSize� chunkOffset�errors� prevNumLines� prevNumCols�_bufferedCharacterr6 r r r rb � s z HTMLUnicodeInputStream.resetc C s t |d�r|}nt|�}|S �zvProduces a file object from source. source can be either a file object, local filename or a string. r4 )rL r �r% rP r" r r r r` � s z!HTMLUnicodeInputStream.openStreamc C sT | j }|�dd|�}| j| }|�dd|�}|dkr@| j| }n ||d }||fS )N� r r r )r* �countrg �rfindrh )r% r- r* �nLines� positionLine� lastLinePos�positionColumnr r r � _position� s z HTMLUnicodeInputStream._positionc C s | � | j�\}}|d |fS )z:Returns (line, col) of the current position in the stream.r )rs re )r% �line�colr r r r$ � s z HTMLUnicodeInputStream.positionc C s6 | j | jkr| �� stS | j }| j| }|d | _ |S )zo Read one character from the stream or queue if available. Return EOF when EOF is reached. r )re rd � readChunkr r* )r% re �charr r r rw � s zHTMLUnicodeInputStream.charNc C s� |d u r| j }| �| j�\| _| _d| _d| _d| _| j�|�}| j rX| j | }d | _ n|s`dS t |�dkr�t |d �}|dks�d| kr�dkr�n n|d | _ |d d� }| j r�| � |� |� d d �}|� d d �}|| _t |�| _d S ) Nrc r Fr r � � � i�� z rl � T)�_defaultChunkSizers rd rg rh r* re ra r4 ri r( �ordrZ �replace)r% rd r8 �lastvr r r rv � s0 z HTMLUnicodeInputStream.readChunkc C s( t tt�|���D ]}| j�d� qd S )N�invalid-codepoint)�ranger( �invalid_unicode_re�findallrf r7 )r% r8 �_r r r r[ s z*HTMLUnicodeInputStream.characterErrorsUCS4c C s� d}t �|�D ]�}|rqt|�� �}|�� }t�|||d � �rrt�|||d � �}|tv rl| j � d� d}q|dkr�|dkr�|t |�d kr�| j � d� qd}| j � d� qd S )NF� r Try i�� r ) r� �finditerr| �group�startr �isSurrogatePair�surrogatePairToCodepoint�non_bmp_invalid_codepointsrf r7 r( )r% r8 �skip�match� codepointr) �char_valr r r r\ # s" �z*HTMLUnicodeInputStream.characterErrorsUCS2Fc C s zt ||f }W nf tyv |D ]}t|�dk s"J �q"d�dd� |D ��}|sXd| }t�d| � }t ||f< Y n0 g }|�| j| j�}|du r�| j| j kr�q�n0|� � }|| j kr�|� | j| j|� � || _q�|� | j| jd� � | � � s|q�q|d�|�} | S )z� Returns a string of characters from the stream up to but not including any character in 'characters' or EOF. 'characters' must be a container that supports the 'in' method and iteration over its characters. � rc c S s g | ]}d t |� �qS )z\x%02x)r| )r �cr r r r H r z5HTMLUnicodeInputStream.charsUntil.<locals>.<listcomp>z^%sz[%s]+N) �charsUntilRegEx�KeyErrorr| r9 �re�compiler� r* re rd �endr7 rv ) r% � characters�opposite�charsr� �regexr; �mr� �rr r r � charsUntil: s0 z!HTMLUnicodeInputStream.charsUntilc C sT |t urP| jdkr.|| j | _| jd7 _n"| jd8 _| j| j |ksPJ �d S r' )r re r* rd )r% rw r r r �ungeti s z HTMLUnicodeInputStream.unget)N)F)r@ rA rB rC r{ r& rb r` rs r$ rw rv r[ r\ r� r� r r r r rN � s & /rN c @ sL e Zd ZdZddd�Zdd� Zd d � Zdd d �Zd d� Zdd� Z dd� Z dS )rO rU N� windows-1252Tc C sn | � |�| _t�| | j� d| _d| _|| _|| _|| _|| _ || _ | � |�| _ | j d dusbJ �| � � dS )rV i �d r N)r` � rawStreamrN r& � numBytesMeta�numBytesChardet�override_encoding�transport_encoding�same_origin_parent_encoding�likely_encoding�default_encoding�determineEncodingr_ rb )r% rP r� r� r� r� r� � useChardetr r r r& � s z HTMLBinaryInputStream.__init__c C s& | j d j�| jd�| _t�| � d S )Nr r} )r_ � codec_info� streamreaderr� ra rN rb r6 r r r rb � s zHTMLBinaryInputStream.resetc C sJ t |d�r|}nt|�}z|�|�� � W n tyD t|�}Y n0 |S rj )rL r r/ r+ � Exceptionr! rk r r r r` � s z HTMLBinaryInputStream.openStreamc C s� | � � df}|d d ur |S t| j�df}|d d ur:|S t| j�df}|d d urX|S | �� df}|d d urt|S t| j�df}|d d ur�|d j�d�s�|S t| j�df}|d d ur�|S |�rnzddl m } W n t y� Y n�0 g }|� }|j �s:| j �| j�}t|t��sJ �|�s$�q:|�|� |�|� q�|�� t|jd �}| j �d� |d u�rn|dfS t| j�df}|d d u�r�|S td�dfS )NrX r � tentativezutf-16)�UniversalDetector�encodingr� )� detectBOMr^ r� r� �detectEncodingMetar� �name� startswithr� �%pip._vendor.chardet.universaldetectorr� � ImportError�doner� r4 r� rG r3 r7 �feed�close�resultr/ r� )r% �chardetr_ r� �buffers�detectorr# r� r r r r� � sR z'HTMLBinaryInputStream.determineEncodingc C s� | j d dksJ �t|�}|d u r&d S |jdv rFtd�}|d us�J �nT|| j d krf| j d df| _ n4| j�d� |df| _ | �� td| j d |f ��d S )Nr rX ��utf-16be�utf-16lerW r z Encoding changed from %s to %s)r_ r^ r� r� r/ rb r )r% � newEncodingr r r �changeEncoding s z$HTMLBinaryInputStream.changeEncodingc C s� t jdt jdt jdt jdt jdi}| j�d�}t|t �s<J �|� |dd� �}d}|s~|� |�}d}|s~|� |dd � �}d }|r�| j� |� t |�S | j� d � dS dS ) z�Attempts to detect at BOM at the start of the stream. If an encoding can be determined from the BOM return the name of the encoding otherwise return NonerW r� r� zutf-32lezutf-32be� N� r� r ) �codecs�BOM_UTF8� BOM_UTF16_LE� BOM_UTF16_BE� BOM_UTF32_LE� BOM_UTF32_BEr� r4 rG r3 �getr/ r^ )r% �bomDict�stringr� r/ r r r r� s& � z HTMLBinaryInputStream.detectBOMc C sV | j �| j�}t|t�s J �t|�}| j �d� |�� }|durR|jdv rRt d�}|S )z9Report the encoding declared by the meta element r Nr� rW ) r� r4 r� rG r3 �EncodingParserr/ � getEncodingr� r^ )r% r# �parserr� r r r r� 3 s z(HTMLBinaryInputStream.detectEncodingMeta)NNNNr� T)T) r@ rA rB rC r& rb r` r� r� r� r� r r r r rO z s � * >"rO c @ s� e Zd ZdZdd� Zdd� Zdd� Zdd � Zd d � Zd d � Z dd� Z dd� Z e e e �Z dd� Ze e�Zefdd�Zdd� Zdd� Zdd� Zd S ) � EncodingBytesz�String-like object with an associated position and various extra methods If the position is ever greater than the string length then an exception is raisedc C s t |t�sJ �t�| |�� �S �N)rG r3 �__new__�lower�r% �valuer r r r� F s zEncodingBytes.__new__c C s d| _ d S )Nr )rs r� r r r r& J s zEncodingBytes.__init__c C s | S r� r r6 r r r �__iter__N s zEncodingBytes.__iter__c C s> | j d }| _ |t| �kr"t�n |dk r.t�| ||d � S )Nr r �rs r( � StopIterationrM �r% �pr r r �__next__Q s zEncodingBytes.__next__c C s | � � S r� )r� r6 r r r �nextY s zEncodingBytes.nextc C sB | j }|t| �krt�n |dk r$t�|d | _ }| ||d � S r' r� r� r r r �previous] s zEncodingBytes.previousc C s | j t| �krt�|| _ d S r� �rs r( r� )r% r$ r r r � setPositionf s zEncodingBytes.setPositionc C s* | j t| �krt�| j dkr"| j S d S d S )Nr r� r6 r r r � getPositionk s zEncodingBytes.getPositionc C s | | j | j d � S �Nr )r$ r6 r r r �getCurrentByteu s z EncodingBytes.getCurrentBytec C sH | j }|t| �k r>| ||d � }||vr4|| _|S |d7 }q|| _dS )z Skip past a list of charactersr N�r$ r( rs �r% r� r� r� r r r r� z s zEncodingBytes.skipc C sH | j }|t| �k r>| ||d � }||v r4|| _|S |d7 }q|| _d S r� r� r� r r r � skipUntil� s zEncodingBytes.skipUntilc C s( | � || j�}|r$| jt|�7 _|S )z�Look for a sequence of bytes at the start of a string. If the bytes are found return True and advance the position to the byte after the match. Otherwise return False and leave the position alone)r� r$ r( )r% r3 r; r r r � matchBytes� s zEncodingBytes.matchBytesc C s< z | � || j�t|� d | _W n ty6 t�Y n0 dS )z�Look for the next sequence of bytes matching a given sequence. If a match is found advance the position to the last byte of the matchr T)�indexr$ r( rs � ValueErrorr� r2 r r r �jumpTo� s zEncodingBytes.jumpToN)r@ rA rB rC r� r& r� r� r� r� r� r� �propertyr$ r� � currentByte�spaceCharactersBytesr� r� r� r� r r r r r� B s r� c @ sX e Zd ZdZdd� Zdd� Zdd� Zdd � Zd d � Zd d � Z dd� Z dd� Z dd� Z dS )r� z?Mini parser for detecting character encoding from meta elementsc C s t |�| _d| _dS )z3string - the data to work on for encoding detectionN)r� r8 r� �r% r8 r r r r& � s zEncodingParser.__init__c C s� d| j vrd S d| jfd| jfd| jfd| jfd| jfd| jff}| j D ]�}d}z| j �d� W n tyx Y q�Y n0 |D ]B\}}| j �|�r~z|� }W q�W q~ ty� d}Y q�Y q~0 q~|sH q�qH| j S ) Ns <metas <!--s </s <!s <?r TF) r8 � handleComment� handleMeta�handlePossibleEndTag� handleOther�handlePossibleStartTagr� r� r� r� )r% �methodDispatchr� � keepParsing�key�methodr r r r� � s4 � zEncodingParser.getEncodingc C s | j �d�S )zSkip over commentss -->�r8 r� r6 r r r r� � s z EncodingParser.handleCommentc C s� | j jtvrdS d}d }| �� }|d u r,dS |d dkr\|d dk}|r�|d ur�|| _dS q|d dkr�|d }t|�}|d ur�|| _dS q|d dkrtt|d ��}|�� }|d urt|�}|d ur|r�|| _dS |}qd S ) NTFr s http-equivr s content-type� charsets content) r8 r� r� � getAttributer� r^ �ContentAttrParserr� �parse)r% � hasPragma�pendingEncoding�attr�tentativeEncoding�codec� contentParserr r r r� � s8 zEncodingParser.handleMetac C s | � d�S )NF)�handlePossibleTagr6 r r r r� � s z%EncodingParser.handlePossibleStartTagc C s t | j� | �d�S )NT)r� r8 r r6 r r r r� � s z#EncodingParser.handlePossibleEndTagc C sb | j }|jtvr(|r$|�� | �� dS |�t�}|dkrD|�� n| �� }|d ur^| �� }qLdS )NTr )r8 r� �asciiLettersBytesr� r� r� �spacesAngleBracketsr� )r% �endTagr8 r� r� r r r r � s z EncodingParser.handlePossibleTagc C s | j �d�S )Nr r� r6 r r r r� s zEncodingParser.handleOtherc C s� | j }|�ttdg�B �}|du s2t|�dks2J �|dv r>dS g }g }|dkrV|rVq�nX|tv rj|�� }q�nD|dv r�d�|�dfS |tv r�|�|�� � n|du r�dS |�|� t |�}qF|dkr�|� � d�|�dfS t |� |�� }|dv �rJ|}t |�}||k�r"t |� d�|�d�|�fS |tv �r<|�|�� � q�|�|� q�nJ|d k�rbd�|�dfS |tv �r||�|�� � n|du �r�dS |�|� t |�}|t v �r�d�|�d�|�fS |tv �r�|�|�� � n|du �r�dS |�|� �q�dS ) z_Return a name,value pair for the next attribute in the stream, if one is found, or None� /Nr )r N� =)r r r )� '� "r ) r8 r� r� � frozensetr( r9 �asciiUppercaseBytesr7 r� r� r� r )r% r8 r� �attrName� attrValue� quoteCharr r r r� sb zEncodingParser.getAttributeN) r@ rA rB rC r& r� r� r� r� r� r r� r� r r r r r� � s $r� c @ s e Zd Zdd� Zdd� ZdS )r� c C s t |t�sJ �|| _d S r� )rG r3 r8 r� r r r r& a s zContentAttrParser.__init__c C s � z�| j �d� | j jd7 _| j �� | j jdks<W d S | j jd7 _| j �� | j jdv r�| j j}| j jd7 _| j j}| j �|�r�| j || j j� W S W d S nP| j j}z"| j �t� | j || j j� W W S t� y� | j |d � Y W S 0 W n t�y Y d S 0 d S )Nr� r r )r r )r8 r� r$ r� r� r� r� r� )r% � quoteMark� oldPositionr r r r� e s. zContentAttrParser.parseN)r@ rA rB r& r� r r r r r� ` s r� c C sb t | t�r.z| �d�} W n ty, Y dS 0 | durZz t�| �W S tyV Y dS 0 ndS dS )z{Return the python codec name corresponding to an encoding or None if the string doesn't correspond to a valid encoding.r N)rG r3 �decode�UnicodeDecodeErrorr �lookup�AttributeError)r� r r r r^ � s r^ )0� __future__r r r Zpip._vendor.sixr �pip._vendor.six.movesr r r� r� �ior r � pip._vendorr � constantsr r r r r rc r r r� r r r � invalid_unicode_no_surrogaterY rm r� �evalr� r� �ascii_punctuation_rer� �objectr! rT rN rO r3 r� r� r� r^ r r r r �<module> sH �� J g Ib ='