msgpack

Provide a `default` function to `msgpack.packb()` that converts unsupported types into a MessagePack-serializable format, often by representing them as dictionaries or using `ExtType` for custom types.

```python
import msgpack
import datetime

def default_serializer(obj):
    if isinstance(obj, datetime.datetime):
        return {'__datetime__': obj.isoformat()}
    # Add other custom types here
    raise TypeError(f"Object of type {obj.__class__.__name__} is not JSON serializable")

data = {'timestamp': datetime.datetime.now(), 'value': 123}
packed_data = msgpack.packb(data, default=default_serializer)
print(packed_data)

# To unpack, you'd need a corresponding object hook or ext_hook
# For simplicity, here's a basic unpack for the example above:
def datetime_object_hook(obj):
    if '__datetime__' in obj:
        return datetime.datetime.fromisoformat(obj['__datetime__'])
    return obj

unpacked_data = msgpack.unpackb(packed_data, raw=False, object_hook=datetime_object_hook)
print(unpacked_data)
```

When dealing with streams or concatenated MessagePack objects, use `msgpack.Unpacker` to process data incrementally, or ensure `unpackb()` receives only a single, complete MessagePack object.

```python
import msgpack

# Example of multiple objects packed together (causes ExtraData with unpackb)
packed_data_stream = msgpack.packb({'a': 1}) + msgpack.packb({'b': 2})

# Incorrect: will raise ExtraData
try:
    obj = msgpack.unpackb(packed_data_stream)
except msgpack.exceptions.ExtraData as e:
    print(f"Caught expected error: {e}")

# Correct: using Unpacker for stream processing
unpacker = msgpack.Unpacker(raw=False) # raw=False for Python strings
unpacker.feed(packed_data_stream)
for obj in unpacker:
    print(f"Unpacked object: {obj}")
```

Ensure the input byte string is a complete and valid MessagePack object. When reading from a stream or network, confirm that all parts of the MessagePack message have been received before attempting to unpack. Use `msgpack.Unpacker`'s `feed` method and iterate over it to handle partial data gracefully.

```python
import msgpack

# Example of incomplete data
incomplete_data = msgpack.packb({'key': 'value'})[:-5] # Truncate 5 bytes

# Incorrect: will raise ValueError
try:
    obj = msgpack.unpackb(incomplete_data)
except ValueError as e:
    print(f"Caught expected error: {e}")

# Correct: When data might be incomplete, use Unpacker
unpacker = msgpack.Unpacker(raw=False)
# Simulate receiving data in chunks
unpacker.feed(incomplete_data)

# The loop won't yield anything until a complete object is fed
for obj in unpacker:
    print(f"Unpacked object: {obj}")

# If later, the rest of the data arrives
remaining_data = msgpack.packb({'key': 'value'})[-5:] # The missing 5 bytes
unpacker.feed(remaining_data)
for obj in unpacker:
    print(f"Successfully unpacked with remaining data: {obj}")
```

When packing, ensure `bytes` objects are treated as MessagePack's binary type by passing `use_bin_type=True` to `msgpack.packb()`. When unpacking, pass `raw=False` to `msgpack.unpackb()` or `msgpack.Unpacker` to automatically decode MessagePack string types to Python `str`, while preserving MessagePack binary types as Python `bytes`.

```python
import msgpack

# Scenario 1: Packing bytes without use_bin_type=True (problematic)
binary_data = b'\x80\x01\x02\x03' # Non-UTF-8 bytes
try:
    # This might pack it as a raw string if use_bin_type is not explicit
    packed_raw = msgpack.packb(binary_data, use_bin_type=False)
    # Unpacking without raw=True (Python 3 default) will try to decode as string
    # and fail if the packed 'raw' contains invalid UTF-8
    # In msgpack 1.1.2, use_bin_type=True is often default/recommended.
    # However, if explicitly set to False during packing and then unpacked without raw=True (default), it can cause this.
    print("Attempting to unpack potentially problematic data...")
    msgpack.unpackb(packed_raw, raw=False)
except UnicodeDecodeError as e:
    print(f"Caught expected UnicodeDecodeError: {e}")

# Correct way: Pack bytes as MessagePack's binary type
packed_correctly = msgpack.packb(binary_data, use_bin_type=True)

# Unpack with raw=False (default for Python strings) to get correct Python types
unpacked_data = msgpack.unpackb(packed_correctly, raw=False)
print(f"Unpacked correctly (bytes object): {unpacked_data}")

# If you specifically want raw bytes for strings as well, use raw=True during unpacking
packed_str = msgpack.packb('hello world')
unpacked_raw_str = msgpack.unpackb(packed_str, raw=True)
print(f"Unpacked string as bytes: {unpacked_raw_str}")
```

Provide a `default` function to `msgpack.packb` to convert unsupported types (e.g., `datetime`) into a serializable format like an ISO string or timestamp.

```python
import msgpack
import datetime

def default_serializer(obj):
    if isinstance(obj, datetime.datetime):
        return obj.isoformat()
    raise TypeError(f"Object of type {obj.__class__.__name__} is not msgpack serializable")

data = {'timestamp': datetime.datetime.now()}
packed_data = msgpack.packb(data, default=default_serializer)
print(packed_data)
```